Data Crow 5.2 Released – Collection Management Software

This is quite a big release. The web client is now a full-featured client. You can search online and add items, lend and return items and use advanced filters. It also has a tree view and can manage templates and property values. Administrators can manage users, modules, backups, server settings and online services from the browser.

Security got a major overhaul. There is two-factor authentication, a password policy, encrypted (TLS) connections between the full client and the server, server-side checks of attachments with an optional virus scan, and a long list of security fixes. It is strongly recommended to change your password(s) after upgrading.

Importing is much improved as well. There is a new generic XML import for files from other collection programs, field mappings can be saved and reused, pictures can be imported from CSV files, and CSV imports are much faster.

Several performance issues have been fixed; the full client scrolls more smoothly and handles items with many child items much better, such as series with hundreds of episodes. Saving items and pictures in the client server mode no longer blocks the application.

And much more;

[New]

Loans; a note can now be added to a loan (when lending an item and when returning it). The note is shown in the loan information and in the loan history, in the full client and in the web client. A loan which has ended (an entry of the loan history) can be changed afterwards: the contact person, the dates and the note (full client: double click on the entry, in the loan history of the item and in the loan administration, or right click and Edit; web client: double click on the entry or the pencil).
Import; new Generic XML File Import, for XML files of other programs (for example an export of another collection program). The items in the file are found automatically and their values are shown as paths (such as genres/genre/name) which can be mapped to the fields of the module, including the Picture fields. Values which occur more than once for an item (such as several genres) are combined. The existing XML import is now called Data Crow XML File Import, as it only reads XML files exported by Data Crow.
Import; the field mappings of the CSV and XML imports can now be saved under a name and selected again in a later import. The settings of the import (such as the separator of a CSV file) are saved with the mapping. The next import also starts with the mapping of the previous import. Columns which are no longer in the file and fields which are no longer available are skipped and reported, new columns can be mapped by hand.
Translations; the resources editor is new in the web client (Settings, Translations, administrators) and has been rebuilt in the full client: one screen for all languages, search in the key, the translation, the description and the language to compare with, filter on topic and on texts which are not translated or have been changed, readable names for modules and fields, replace all in the texts shown, and download or load a language file. A text must be filled and must keep the parameters of the English text. English can no longer be changed: it is replaced at every upgrade.
Two-factor authentication: users can switch on a second step at login in the web client (Settings, Change password) or the full client connected to a server (User menu) with any authenticator app on their phone. The full client and the web client ask for the code after the password. With a checkbox the code is not asked again on that device for 30 days; the password is always needed, and changing the password or Forget all devices in the settings ends this. There are recovery codes for when the phone is not at hand, administrators can switch it off for a user in the user management, and the server option -reset_2fa: does this for the last administrator. Users who do not switch it on do not notice anything. Apps which do not support it yet, such as older versions of the Data Crow Reader, cannot log in with a user who has switched it on.
Security; new Security settings with a password policy: a minimum password length and a complexity level (none, letters and digits, upper and lower case and digits, or also special characters). Users whose password does not meet the requirements have to choose a new password when they log in, in the full client as well as in the web client. Passwords can also be set to expire after a number of days (0, the default, means never).
Security; the maximum size of attachments has moved from the size limits to the new Security settings. The Security settings also hold the allowed or blocked file types of attachments, a check of the content of attachments (programs and scripts are refused under any file name, as are files which do not match their file type) and an optional virus scan with ClamAV (installed separately, reached over the network). All checks are done by the server, for the web client as well as the full client.
Client-Server; administrators can now create and change modules from the full client (Modules menu): the Create Module Wizard, the Alter Module Wizard (fields can be added, renamed, removed and converted, and the texts, options and icons of the module can be changed), the Copy Module Wizard and the Relate Module Wizard. With Add & New several modules can be defined before anything is sent to the server, and the next module can refer to them (for example a property module and the main module which uses it); with Send to Server all modules are created in one go: either all modules are created or none. The server checks the changes, makes a snapshot of the data first (kept in the backups of the server) and restores the previous situation when something goes wrong; a progress bar is shown meanwhile. The server cannot be used by others while it makes the change. The web client shows the changes right away; the full client needs to be restarted.
Client-Server; administrators can now make a backup on the server from the full client (Tools menu, Server Backup): a full backup or a backup of the data only (no pictures and attachments). The backups are kept on the server and can be restored and deleted from the same screen. Before a restore the server checks the backup (version and damage; a damaged backup changes nothing) and makes a snapshot of the current data. Then the data, settings, modules, pictures and attachments of the backup are put back (what was added after the backup is removed, but a backup of the data only leaves the pictures and attachments as they are) and loaded again without restarting the server. The server cannot be used by others while a backup is made or restored; they get a message that the server is in maintenance mode. All users are logged off when a restore is finished, and the full client needs to be restarted.
Client-Server; administrators can now change the settings which apply to all users from the full client: the Security settings, the checks for required fields and unique values and the field settings of the modules (enabled, required, unique and so on). Other users see these settings but cannot change them. Numeric settings are checked against the values which are allowed.
Client-Server; administrators see a Sessions tab in the log window of the full client, with the connections to the server and a button to close a connection.
Backups of the server can be scheduled on the backup screen of the web client and of the full client (new tab Scheduled backups): daily or weekly at a chosen time, with the number of scheduled backups to keep. The time is entered in your own local time, also when the server is in another time zone, and stays the same when summer time starts or ends. A scheduled backup is a full backup; the server cannot be used while it is being made. Only scheduled backups are removed automatically.
Server; the connection of the full client with the server can be encrypted (TLS). Start the server with -tls: the server creates a certificate for itself (in the security folder of the user directory) or uses your own (-tls_keystore and -tls_keystore_password_file). The login screen of the full client has the new option Encrypted connection (TLS); the first time the user is asked to trust the fingerprint of the certificate, a different certificate later on is reported as a warning. The server then only accepts encrypted connections. The image server uses the same certificate on its own port and accepts encrypted (full client) and plain (web client behind a proxy) connections; the full client loads the pictures encrypted. The full client warns when you connect to a server on another computer without an encrypted connection (you can choose not to be warned again for that server). A full client which does not use TLS gets a message after a few seconds that the server needs an encrypted connection. See documentation/data_crow_howto_tls.md.
Web; new section Module wizards in the settings (administrators): create modules and change the modules you made yourself, as the module wizards of the full client do. Choose the type of module, give it a name, options and icons, and define its fields (type, reference to another module, searchable, required, descriptive, key field, tab). With Add & New several modules can be defined, which can refer to each other, and Send to Server creates them together. The module wizards (web client and full client) have one icon field: choose an image of any type and it is stored as an icon of 16×16 and of 32×32 pixels. The field types are sorted alphabetically (web client and full client), the fields every module of a type has are shown for information in the fields step, and after a change the labels are retrieved from the server again, so new modules and fields are shown with their names right away. Existing modules are shown as in the full client: a main module with the modules it refers to below it (open a module with the arrow, or use the filter). Existing modules can get new fields, fields can be renamed or converted, and fields you added yourself can be removed. The server makes a backup first and undoes the change when something goes wrong; users who are logged in see the new modules after they log in again. Modules cannot be removed in the web client. The wizards Copy Module (a new module based on an existing one) and Relate Module (make a module the child of another module, such as episodes of a series; a warning with the parent and the child has to be confirmed first, and the relation can only be undone by restoring a backup) are available there as well.
Web; the settings have a new section Enabled Modules (administrators): choose which modules are available for all users, the full client connected to the server follows this choice and administrators can change it there as well (Active Modules in the settings). Users see the change after they log in again.
Web; settings: a new settings screen (the gear button at the top) with the sections on the left and the options of the selected section on the right. The search box at the top finds sections and options (such as a setting of the server) and takes you to them. On a phone or tablet the sections are a list above the options. The choice of the theme (light, dark or same as the device) is in the section Appearance; same as the device now follows the device, also when it changes.
Web; administrators can now change the settings which apply to all users (such as the Security settings) and the field settings of the modules (enabled, required, unique and so on) in the web client (Settings).
Web; administrators can now manage the users in the web client (Settings, User management): create users, change their name, email address, administrator rights and whether they can log in, set a password, delete users, and set per module and per field whether a user may view or edit, with the same actions as the full client (allow viewing of all fields, allow editing of all fields, deny module access). Modules which are shared by other modules, such as the tags, are listed once. Changes apply to users who are logged in right away. The permissions of the plugins (which tools a user may use) can be set here as well.
Web; administrators can now make backups on the server in the web client (Settings, Server Backup): a full backup or a backup of the data only, the same as in the full client. The backups on the server can be downloaded, restored and deleted from the same screen. The browser downloads a backup itself, so backups of several GB can be downloaded; the download is written to the server log. While the server is in maintenance mode (for example while a backup is made) the login page tells that and shows the login again when the server can be used; users who are logged in see a message at the top of every page until then.
Web; administrators can now see the server log and the connections in the web client (Settings, Event Log and Sessions). The Event Log shows the latest entries of the server log, filtered on level and text, refreshed automatically, with the stack traces of errors; the log file can be downloaded. Sessions shows who is connected (the full client and the web client): select a connection and close it. Both can also be used while the server is in maintenance mode.
Web; new System Information page in the settings (administrators): the version of the server, when it was started and how long it has been running, the number of connections, the installation and data folders, the database, the memory (maximum, used and available), the Java version and the operating system. The page updates itself. The number of connections is a link to Sessions, and the date of the last full backup is shown with a link to Server Backup.
Web; users can now change their own password in the settings (Change password). The current password has to be entered as well, and the new password has to meet the password policy of the server.
Web; new section Data Crow Reader in the settings, for all users: scan the QR code with the Data Crow Reader app to connect it to the server, instead of typing the server address. Below the QR code are the settings to enter in the app by hand (HTTPS, server address and port).
Web; new section Online services in the settings (administrators): the version of the online services pack, checking for a newer pack (downloaded and loaded without restarting the server) and the settings of the services, such as the API keys of Google Books, TMDB, OMDb, RAWG, Mobygames, Last.fm, Comic Vine and the AI search. The online search of the web client uses these settings of the server; the full client has its own settings. Per module you can also choose which fields are filled with the result of an online search, whether existing values may be replaced and which fields may replace them. The server checks for a new online services pack when it starts; this can be switched off in the server settings. A message of a service, such as the Google Books limit which all users share until you have set your own API key, is now shown in the online search.
Web; online search (the new button in the top bar of the overview, for modules which have online services and for users who may add items): search with one of the online services of the module, choose the region, search mode and filters, and see the results (with picture) coming in. Clicking a result shows a preview next to the results (the values and the pictures). Select one or more results and add them as new items. In the edit page of an item the same button (Online Search) searches for the item and updates it with the chosen result: empty fields are filled (filled fields are only replaced when the module is set to overwrite) and the pictures of the result are shown under Pictures, after which you can change the item before saving it. When the result would not change the item a message explains this. The chosen service, region and search mode are remembered per module on the server, so the choice is the same on every device. The server does the searching with the same online services as the full client; it loads the services pack when it starts (and downloads it when it is not installed yet, so the server needs an internet connection for that). The server installation now contains the online services pack.
Web; templates: new screen to maintain the templates of a module and of its child module, such as tracks (administrators; button Templates in the top bar of the overview): create, change and delete templates, give them a name and mark the default template. When a new item is created the template can be chosen (the default template is chosen at the start) and fills in the form.
Web; Edit as New: an item can be copied with its values, pictures, attachments and child items (such as the tracks of a music album). For modules with child items of the same module (such as editions) a copy of an item can be added as a child item.
Web; loans: items can now be lent and returned in the web client (the section Loan Administration of an item): lend to a contact person with a start date, a due date and a note, return the item and see the loan history. Several items can be lent or returned at once: the button Select items in the top bar of the overview shows a selection bar; select the items and use Lend Items or Return Items (one person, dates and note for all of them; items which have been lent already, or which are not lent, are skipped and named). A container is lent and returned with the items in it, as in the full client. The new screen Loan Administration (button in the top bar of the overview) shows the loans of all items, filtered on current loans, the history, contact person, dates and late returns only, the same as in the full client. Users need the permission to lend (the Loan permission).
Web; new screen to maintain the values of the property modules (categories, storage media, tags and so on) which the module in the overview uses (the new tags button in the top bar): one section for each property module, with add, edit, create multiple (one value per line, values which exist already are skipped), merge (the references of the selected values are moved to the target, which keeps their names as alternative names) and delete (a value which is in use is only deleted after the references have been removed, with a warning). The list shows how often each value is used.
Web; tree view in the overview (the new tree button in the top bar): group the items by up to three fields, for example by year and then by genre. Every value shows the number of items, a click on a value shows the items of that value and a second click removes the selection. Dates are grouped by year, and items without a value are in the Empty node. The selection is remembered per module.
Web; advanced filtering in the overview (the new magnifier button in the top bar): a filter can have several conditions on the fields of the module, each with an operator which fits the field (equal to, contains, empty, before, after, greater than and so on), combined with And or Or, with a sort order. Filters can be saved with a name, edited and deleted; they are saved per user on the server, so they are available on every device. Saved filters are shown above the overview: a click applies a filter, a second click removes it. The active filter is shown as a tag with the other filters.

[Changed]

Editions (items which belong to another item of the same module, such as the editions of a book) are shown with their picture on the left and one or more lines of text: by default the title and, below it, the description. You choose which fields are shown and on which line with the settings button above the list; the web client (administrators, for all users) and the full client each have their own setting. Web; the editions can be filtered with the filter above the list and are shown in pages of 30. After opening an edition, going back shows the editions again (same page and filter) with the edition you opened in view. Full client; after saving an edition the quick view of the main window shows the change right away.
Saving an item with many child items (such as a series with many episodes) shows a progress bar in the item form and when items are added from the online search (full client). The child items of a new item are saved without checking them one by one; they can be corrected when they are edited.
Added edit as new and create to the editions panel on the item form.
Loans; an item which has been lent now shows a coloured bar at the top (to whom, since when and when it is due; red when it is overdue), in the full client and in the web client. For a contact person the loan administration opens filtered on that person with the history and the current loans (full client: button in the loan tab of the contact person; web client: the Loan Administration section of the contact person). The card view of the full client shows a small label (Lent or Overdue) on the items which have been lent, as the overview of the web client does.
The filters of the loan administration (full client) have a better layout: the labels take the room they need and the fields use the rest, so the fields on the right do not become small when the window gets smaller.
Import; pictures can now be imported from a CSV file. Map a column with the path of a picture file (or a web address) to Main picture or to Other pictures (the field mapping shows these two instead of Picture 01 to Picture 10). The main picture comes first and is the picture shown in the overview; the other pictures follow in the order of the columns. Several columns can be mapped to Other pictures. A path which does not exist on this computer (for example from an export made on another computer) is also looked for in the folder of the import file and its subfolders, such as Images/1.jpg. Before, picture columns could not be selected in the field mapping. A column can hold several pictures, separated by | or on separate lines, and a picture can also be included in the file itself as base64 data (with or without data:image/…;base64,). Web addresses with spaces are loaded as well. The names of picture files and their folders are matched ignoring upper and lower case, also on Linux (Images/Cover.JPG finds images/cover.jpg). The type of a picture is taken from its content, not from its name: a WebP or PNG picture called cover.jpg is imported, with a message that its name does not match its type; this also applies to WebP pictures on the internet. A file which is not a picture is skipped with a message, and a damaged picture is imported as far as it can be read, also with a message. These messages are shown in the import and written to the log. The same applies to the pictures of the Generic XML File Import. Files and saved mappings which use Picture 01 to Picture 10 still work: Picture 01 becomes the main picture, the others become other pictures.
Export; the CSV export writes the pictures as two columns, Main picture and Other pictures (all other pictures, separated by |), instead of a column per picture field, so pictures after the 10th are no longer left out. Selecting Picture 01 exports the main picture, selecting one of the other picture fields exports the other pictures. Importing the file again restores all pictures in the same order.
Import; importing from a CSV file is much faster. Before, the import waited one second for every item.
Import; the CSV import recognizes the character set of the file itself (UTF-8, UTF-16 and Windows files, such as an export of Collectorz), so the character set setting has been removed.
Import; the separator of a CSV file is detected automatically and can be chosen from a list (comma, semicolon, tab or vertical bar) instead of being typed.
Import; a CSV file without a header row can be imported: switch off the option that the first row contains the column names. When no column could be matched to a field a warning is shown. Column names are matched ignoring upper and lower case and spaces, and in the names of the fields in all languages. Columns with the same name or without a name no longer shift the other columns.
Import; empty lines in a CSV file are skipped.
Import; a list in a single value of a CSV file (such as the genres or actors of an item) is detected: it can be separated by | or ; or be on separate lines, else by commas. Names such as “Allen, Woody” stay together.
Import; numbers (such as 1,5, 1.5 or 1.234,56) and dates are read in the notation used in the file: dates as 2002-05-17, 17/05/2002 or 5/17/2002 (the order of day and month is taken from the other dates in the column), and with the names of the months in all languages of Data Crow (such as 17 mei 2002 or May 17, 2002). A value which cannot be read gives a short message in the import instead of an error with technical details in the log.
Import; the Generic XML File Import and the Data Crow XML File Import read XML files more reliably: other character sets (such as UTF-16) are supported, a file which cannot be read gives a message instead of an import which does not finish, and the files are read more securely (external entities are not loaded). Lists, numbers and dates are read as in the CSV import.
Modules; the standard fields of a new module (such as Title, Description, Year, Languages, Rating and Countries of a media module) have a translation in all languages right away: the translations are copied from the modules which come with Data Crow and are of the same type. This applies to the web client, the full client and the server; the translations you have made yourself are never replaced. The full client connected to a server adds the missing translations of the modules of the server when it starts
Modules; the Alter Module Wizard now shows the modules as a tree: the main modules, below them the modules their fields refer to and below those the modules these refer to. Modules which are not used yet are shown separately. A filter and a double click to continue were added. The Alter Module Wizard no longer has a Finish & New button, and the Finish & New button of the Create Module Wizard and the Copy Module Wizard is now called Add & New.
Templates; in the full client the form of a new item has a drop down with the templates on top (the default template is selected, choosing another template fills in its values).
Online search; the pictures of child items (such as the episodes of a series) are no longer retrieved unless you ask for them: retrieving them made a search for a series with many episodes take very long. Online services which can retrieve them (for now The Movie Database (TV Series)) have the new setting Retrieve the pictures of the child items, off by default (full client: the settings button next to the service in the online search; web client: Settings, Online Services).
Online search; Open Library: searching on a title or keyword is much faster and no longer retrieves all editions of all books before anything is shown. It shows the best matching edition per book, in the chosen language, and the results come in one by one. Covers and the year of first publication are now filled in, and only the language of the edition is set.
Full client; opening an item and Edit as New show a progress bar when retrieving the item (or copying it, with its child items and pictures) takes longer than a moment, instead of Data Crow not responding.
The card view in the full client now scrolls more smoothly: the items and pictures are retrieved in the background instead of one card at a time while scrolling, in the standalone mode as well as in client-server mode.
The table view in the full client now scrolls more smoothly: the rows are retrieved in the background instead of one at a time while scrolling, in the standalone mode as well as in client-server mode.
The splash screen is now shown right after starting Data Crow. Before, it could take several seconds before it appeared.
Full client connected to a server; the item form opens faster, also on a slow connection: the related items and the loan of the item are retrieved together with the item, the pictures, the attachments and the child items are retrieved in the background and shown when they are there (also in the quick view), and the Loan tab is filled when it is opened.
Full client connected to a server; the translations are those of the server: the resources editor changes the texts on the server and the full client retrieves them when it starts
Full client connected to a server; the Synchronization Service now shows the QR code of the server the full client is connected to, so the Data Crow Reader app connects to that server. No service is started on your own computer anymore. Below the QR code are the settings to enter in the app by hand (HTTPS, server address and port); the standalone full client shows these as well.
When a task fails because the connection to the server was temporarily broken it will now no longer hang the system indefinitely. Timeout has been set to 30 seconds, after which the action can be retried.
Client-Server; new items are saved first and their pictures are sent afterwards, one by one. Before, an item was sent with all its pictures in one go. This speeds up saving.
Server; a login of the web client and the Data Crow Reader app now also ends after a time without activity, after which the user has to log in again. Both times can be set in the server settings (Security): after how many minutes without activity (5 to 10080, by default 240) and after how many hours at most, also when the user stays active (1 to 720, by default 24).
Timeouts in the connection to a client (Data Crow Server) are now logged without a stack trace.
Server; addresses that repeatedly fail to connect to the client port of the server (failed logins, or programs that are not a Data Crow client such as port scanners) are now blocked for 60 minutes after 5 failed attempts within 10 minutes. Addresses in the local network are never blocked.
Server; the same protection now applies to the login of the web client and the Data Crow Reader app. When the server is behind a reverse proxy, start it with -trusted_proxies: so the real address of the user is used.
Server; users without a password can now only log in from the local network. The automatic login of the web client (when only the SA user without a password exists) also only works from the local network. Set a password for users who connect from outside the local network.
Server; the maximum size of attachments is now also checked by the server (also for the full client). Pictures larger than 50 MB or 50 megapixels are refused.
Server; the -bindto startup option now also applies to the API server and the image server. Without it the web, API and image server listen on all addresses (0.0.0.0); before, the API and image server only listened on the address of the host name, so pictures were not shown when a client connected through another address such as localhost. With -bindto:127.0.0.1 the web, API and image server can only be reached through a reverse proxy on the same machine.
Server; the full client and the server must now have the same version. When the versions differ, logging in is refused with a message that tells which version to install, instead of the client partly working (for example without pictures).
Server; new startup option -appserver_bindto to choose the address the application server (used by the full client) listens on. With -appserver_bindto:127.0.0.1 the full client can only connect from the server itself, for example through an SSH tunnel, so the application server cannot be reached from the internet.
Server; the server no longer logs in as a user when it starts. The credentials (-credentials or the datacrow.credentials file) are only used to create the first administrator when the database has no users yet; after that they are not needed and ignored, so they can be removed from the start command. Changing the password of that user no longer prevents the server from starting.
Server; the internal administrator account of the server can no longer be used to log in.
Web; the child items of an item (such as the episodes of a series) are shown in pages of 50, with a filter above them when there are more, when viewing and when editing the item. They load much faster: before, all child items were sent at once, each with its pictures and related items. Server; new REST service /children/{module}/{item}/page with start, count and filter; /children/{module}/{item} is unchanged.
Web; a language which was added on the server can be chosen at the login.
Web; the overview shows the number of child items (such as editions) on the items which have them, as the card view of the full client does.
Web; the Home button is always shown in the top bar (right of the buttons of the screen, except on the overview).
Web; the top bar has a new look: the Data Crow icon at the far left, with the name and the version of Data Crow under it, and the buttons in a rounded group on the right. On a phone the buttons are shown under the icon.
Web; numbers and dates can now be searched with comparisons in the search field: choose the field and enter for example >2000, <=1999 or =2005, or several for a range (>1990 <2000). Dates can be entered as 2020-01-31 or 31-01-2020; a year alone (>2000) stands for the whole year. The search field shows an example for these fields.
Web; the file of a file field (the view of an item) is now downloaded by the browser itself: the file is saved while it comes in, with the progress in the browser, and large files (several GB) can be downloaded without using the memory of the browser. A download can be continued. Clicking the file name opens pictures, pdf, plain text, audio and video in the browser (video can be skipped through); other files are downloaded.
Web; an attachment of an item can now be opened in the browser by clicking its name (pdf, pictures, plain text, audio and video). Other files, such as html files, are downloaded. The download button works as before.
Web; viewing, editing and creating an item now have the same layout as the settings: the sections of the item (Details, Pictures, Attachments and so on) are on the left and the selected section is on the right. On a phone or tablet the sections are a list above the content. Viewing an item has a new Pictures section which shows all pictures as a list (the carousel in the details stays). Changes in the form are kept when you look at another section. The button for the field settings of an item now has a sliders icon, so it is not confused with the settings of the application (gear).
Web; the web server now sends security headers which prevent the browser from running scripts from unknown sources.
Web; Bootstrap and the Bootstrap icons are now part of the web client instead of being loaded from an external website.
Web; when the web client is used over HTTPS, browsers are now told to always use HTTPS for it (HSTS). Behind a reverse proxy this requires the -trusted_proxies startup option.
Web; updated libraries.

[Fixed]

Data Crow could keep running in the background (visible in the Task Manager) after it was closed. If a step of closing Data Crow failed, such as saving the settings or closing the database, the program did not end. Every step is now logged when it fails, and Data Crow always ends, also when closing the database takes too long.
Item form; fields at the bottom of a tab (for example the last fields of a module with many fields) were cut off and could not be reached when the form was not high enough. Every tab now has a scrollbar when needed.
Web; reloading the page in the browser logged you out. You now stay logged in as long as your session on the server is still valid.
Import; a CSV file with a line break within a value between quotes (such as a description of several lines) was read wrongly: the rest of the value became a new item. Such values are now read as one value, with the line breaks kept.
Export; for an item with 11 or more pictures the CSV and XML export wrote the wrong picture for the Picture 01 field (the 11th picture instead of the first one).
Copying an item and adding it as an edition (full client and the plugin Copy Item and Add as Child) gave the copy empty attachments: the attachments are now copied with their content. Edit as New in the full client did not copy the attachments at all; it now does.
TV episodes were sorted wrongly when a season has 100 episodes or more: S01E100 came before S01E11. They are now sorted on the season and the episode number, in the full client and the web client.
Searching in text fields did not find values with an underscore or a percent sign in them: for example, searching for client_zipped in the file name did not find datacrow_5.2.0_client_zipped.zip. This applies to the search (contains, starts with, ends with, search in all fields) and to the filter with equal to, in the web client and in the full client.
Filter; when a saved filter was edited (Filter, Manage, Edit) in another language than English, the conditions which were combined with Or came back as And. The Or is now kept.
The lists to choose a field from (the field mapping of the import, the filter, find and replace, the quick filter of the tool bar, charts and renumbering) were not sorted alphabetically in other languages than English and ignored changed field labels. They are now sorted on the label as shown.
Invalid modules, created in older versions, could cause Data Crow not to start. These modules are internal to Data Crow and should never have been saved to disk. Data Crow now checks for them at startup and removes them.
Data Crow; when the credentials which were supplied at startup were no longer correct (for example after changing the password) starting failed with an error. The login screen is shown instead.
Import; in the CSV import, fields which were selected by hand in the field mapping were not imported; only the fields which were matched automatically were used.
Import; a CSV file from Collectorz (UTF-16) could not be imported: the text was unreadable.
Import; when the title field of the module is a number or a date, the import logged an error for every item when existing items were matched and merged.
Online search; when a service answers that something does not exist the search simply finds nothing, instead of showing a technical error. API keys and secrets no longer appear in these messages and in the log.
Online search; TheTVDB, The Movie Database, Discogs, MovieMeter and BoardGameGeek use a key which Data Crow downloads from datacrow.org at startup. When this failed the search reported an error about the settings of the service; a clear message now explains what to check.
Online search; when a request to an online service fails a clear message is shown for the error code (for example that the API key is not valid, that the limit has been reached or that the service has a temporary problem), with what you can do about it. The complete answer of the service is written to the log (without the API keys). Google Books tries a request again (three times) when Google answers that it is temporarily unavailable (503).
Online search; Discogs: a search by release ID did not fill all fields.
Full client; the tracks of an album and the episodes of a series in the item form used a fixed width: the title now takes the width of the window and the play length is shown at the right side.
Full client; opening the tab with the child items in the item form (such as the episodes of a series) made Data Crow hang for a while: all child items were loaded at once. Only the child items which are visible are loaded now, the filter above them no longer hangs on the first letters (the names are retrieved in one go), the episodes are in the right order (S01E11 before S01E100) and the quick view of the main window is no longer loaded again when the form opens.
Full client; selecting an item with many child items (such as a series with hundreds of episodes) made Data Crow hang for a long time: the quick view loaded the references (such as the languages and the actors) of every child item one by one. The quick view now shows the first 100 child items, followed by …, and loads the references of these only.
Full client; saving an item with many child items (such as a series with many episodes) froze Data Crow for a long time and could stop with an out of memory error, in the item form and when items were added from the online search.
Full client; when an item could not be saved, the buttons of the item form stayed disabled.
Full client; starting Data Crow with -userdir still asked for the user folder: with a new folder the User Folder Configuration window was shown, otherwise the question to change the user folder. The folder given with -userdir is now used (and set up when it is new) without asking.
Full client; closing the User Folder Configuration window with the close button left Data Crow running in the background. At startup without a user folder Data Crow now closes; when changing the user folder it continues with the current folder.
Starting the full client with an installation which does not contain the online services (the services folder) caused an error in the log, and the online services could not be used. The client now starts without them and, when the automatic update of the online services is on, downloads the services.
Full client; fields with multiple references (such as categories) could not be changed in the table view and were shown without their icons. A double click on the cell now opens the selection window, as in the item form, and the values are shown with their icons.
Full client connected to a server; adding pictures or attachments to an item (in the item form and in the quick view) and saving an edited picture made Data Crow hang until they had been sent to the server. They are now sent in the background with a progress bar, and several pictures or attachments are sent in one go.
Client-Server; the dates of Created and Modified were saved without the time.
Client-Server; creating, changing and removing users through the server was not handled correctly. These actions are now handled by the server and are only allowed for administrators.
Server; starting the server with a new user folder logged an error because it tried to create the already existing SA user.
Server; an UnsupportedOperationException was logged on startup when no installation folder (-dir) was specified.
Server; starting the server with a new user folder overwrote the user folder setting of the Data Crow desktop client (datacrow.properties).
Server; the reference list of a module failed to load when the icon file of one of its items was missing.
Server; icons could fail to save with an access denied error when several requests loaded the same icons at the same time.
Server; security fix: passwords of users could be changed through the client port of the server without logging in. Changing a password now requires a login; users can only change their own password, administrators can change all passwords. Please change your passwords after upgrading.
Server; security fix: the server now only accepts valid Data Crow requests on the client port and rejects other data, which could otherwise be used to run code on the server.
Server; security fix: logged in users of the web client could download any file from the server. Only files of a file field of an item can be downloaded now.
Server; connections from port scanners and other programs that are not a Data Crow client were kept open and in memory by the server.
Server; security fix: the rights of users were only checked by the web client itself. The server now also checks them: users can only view and edit items of modules (and fields) they have access to, deleting items requires the delete right, editing pictures and attachments requires the corresponding rights, and only administrators can change the field settings of the web client.
Server; security fix: uploading an attachment could write files outside the attachment folder of the item.
Server; security fix: users of the full client could get administrator rights on the database through SQL statements. Statements that change users or rights are now only allowed for administrators.
Server; security fix: pictures and icons could be viewed by anyone who knew or guessed their address, without logging in. Pictures are now only shown to logged in users; picture addresses handed out to the web client are valid for a limited time (and until the server restarts).
Server; the image server also listened on an additional, random port on all network interfaces.
Server; when the session of the full client had expired (after 24 hours) the server started a new session for every request of the client, which kept using more memory. The session is now renewed instead.
Server; security fix: restoring a (tampered) backup could write files outside the user folder. These files are now skipped.
Server; security fix: pictures, attachments, child items and references could be viewed by users without access to the module of the item.
Server; security fix: the full client could store attachments and pictures outside the attachment and picture folders by using a manipulated file name.
Server; the security token of a client was written to the log file when an error occurred.
Server; security fix: users who are not an administrator could make themselves administrator or change the rights of other users (through the full client, SQL statements or the API of the web client). Users and permissions can now only be changed by administrators. Existing databases are corrected when the server or Data Crow starts.
Server; the log showed an error with a stack trace for every item without a picture when pictures were retrieved over the network.
Server; the web client could not log in when the only user was not SA (for example the first administrator created with -credentials): the automatic login without a password was used. It is now only used when SA is the only user.
Server; unexpected errors in the API only resulted in an error for the web client and were not written to the log file. They are now logged.
Web; the child items of an item (such as the tracks of an album) could be empty after a child had been added, until the page was loaded again: the browser reused an earlier answer of the server. The server now tells browsers not to keep answers of the API.
Web; the child items of an item (such as the tracks of an album) showed an extra first column with the ID of the parent item for child modules which were made with the Relate Module Wizard; the column is not shown anymore.
Web; the overview settings (Overview field settings) gave an error for modules of which fields had been removed or which were created or changed with the module wizards.
Web; downloading the file of a file field (the download button in the view of an item) did not work for file names with a percent sign or a plus sign.
Web; after a field had been removed with the Alter Module Wizard, the screens to edit and create an item no longer worked.
Web; when a reference field pointed to a module which was not enabled, the create option of the drop down showed an error.
Web; the selected field in the filter is no longer deselected when a filter is removed.
Web; after a failed upload of a picture or attachment the busy message stayed on screen.
Web; the hours, minutes and seconds of an existing duration (runtime) were shown as 0 when editing an item, and some durations lost a second.
Web; a date could not be removed from an item.
Web; reloading the page while editing an item showed an empty page instead of the login screen.
Web; a failed login was only reported after a delay, and a slow successful login could briefly show a login failed message.
Web; downloading the file of a file field failed when the API runs on a different address or port than the web client.
Web; attachments with special characters, such as # or ?, in the file name could not be downloaded or deleted.
Web; labels without a translation were shown as “undefined”.
Web; the clear and remove buttons of the search filters were not translated.
Web; security fix: text containing HTML (for example descriptions from online services) could run scripts in the browser. Unsafe content is now removed and only long texts can contain formatting.
Web; security fix: logging out did not end the session on the server; the session could still be used afterwards.
Web; the tooltip of the home button showed the label key instead of a text.
Web; fields without field settings (for example fields which were added after the settings were saved) could cause errors in the web client.
The picture preview in the file selection window (for example when importing pictures) did not fit in the preview area: larger pictures were cut off. The picture is now scaled to fit, also when the window is resized. Browsing through large pictures also no longer builds up memory use, and a file which is not a readable picture no longer causes an error in the preview.

Data Crow 5.2 has been released