Changelog

In December 2022, I published a blog article introducing Datablist for Lead Management. I shared a preview of the next features to be developed on Datablist.

In January, the first iteration was released with tons of new features!

New header

A new collection header was released in January. It is responsive and easier to read.

It is split into 3 parts: Collection Information, Search and Filters, and actions. Filters can be saved with "Saved Filters" (more about that later).

Actions are grouped in 4 menus: Import, Export, Clean, Enrich, and Edit. Read our Datablist for Lead Management post to learn more on that.

The columns headers have been improved. The items count is available on the top right, and a menu to manage a property pop up on a column click.

A "New Property" shortcut has been added near the right of the columns.

Clean and Edit features

Cleaning and editing data is a strong focus for 2023 (see our Datablist 2022 in review blog post).

In January, I added two long awaited features: Find and Replace and Bulk Edit. You can read the Find and Replace documentation and the Bulk Edit documentation to learn more.

DataType conversion

A core concept with Datablist is to work with data types. In spreadsheet tools, data is mostly text with some "formatting". Working with date, boolean, number, etc. is a pain in Google Sheets or Microsoft Excel.

A new DataType conversion tool has been released in January to quickly create Datetime, Number, and Checkbox properties from a Text property. Check our Text to Datetime, Number, Checkbox documentation.

This tool will be improved to convert data from and to any DataType.

Import from collection

Import data into your collection from another collection. With this feature, it is now easier to have segmented lead lists and build a master list.

Join on a property during the import

Consolidating data is an impossible task on spreadsheet tools. It was previously possible in Datablist with the "unique values" option in a property.

Joining CSV files or two collections to consolidate with an identifier is now accessible directly during the import process.

When importing data into an existing collection with data, a new "Join" toggle is available on the mapping properties.

Then, in the options, you can select LEFT OUTER JOIN or FULL OUTER JOIN to import only matching items or all the items from the import file.

Saved Filters

Creating segments in a collection is now possible with "Saved Filters". This feature is accessible when you have at least one filter enabled on your collection. Click "Save Filters" to save them in your account. Saved Filters are shared with your team members.

Miscellaneous

• Use the import file name to rename the collection after an initial import
• Option to prevent duplicating data during a collection cloning
• Duplicate property with an option to copy values
• Show "tooltip" with editing how-to after double-clicking on a cell
• Add keyboard shortcut to close modal with Escape on Delete Collection and Duplicate Property modals
• Improve copy-pasting behavior
• Disable horizontal over-scroll on datatable to avoid the browser previous page behavior

Fixes

• Fix deduplication algorithm on multi-properties analysis. The bug led to misses in the duplicate check.

Filtering with "or" operation

You can now change how multiple filters are combined.

Select the "or" operation to get items matching at least one filter.

Select the "and" operation to get items matching all the filters.

Currently, only one operation can be used for all filters.

Create a new collection from selected items

We added a shortcut to creating a new collection with items. The action is available from the "selected" items actions.

It clones the current collection properties and copies the selected items in it.

Auto-Detect CSV file encoding

The CSV format is text-based but the encoding is not standardized. Two CSV files generated by Microsoft Excel can use two different encodings if one is generated from a system configured in French or in English.

And the bad news, the encoding is not stored in the CSV format.

When loading a CSV file, Datablist has used UTF-8 encoding so far. It worked for English-based CSV files. Or if by chance the CSV file was encoded in UTF-8.

For CSV files with accents or special characters, you ended up with weird characters.

Datablist now analyzes your CSV file to list potential encodings. A score is calculated for each encoding and the one with the highest score is used.

You can change the encoding if the imported data still contains weird characters.

Fixes & Improvements

• Fix cloud synchronization errors after "undoing" items delete
• Fix Excel file import when header names are numbers
• Fix cloning empty collection
• New "is not" and "does not contain" filters

Combine or drop conflicting values when merging duplicate items

In September and October, we improved the auto-merge algorithm in our "Duplicates Finder". The previous version could only merge non-conflicting items. For example, if two items were duplicates on their email address. Datablist could merge them if the other properties had similar or empty values.

All the remaining items with conflicts had to be merged manually.

On CRM data or any contacts list, most of the duplicates found had to be manually merged.

With our improved auto-merging algorithm, you can now define how to deal with your conflicting data during the merge process. Two options are available :

• Combine conflicting properties - With this option, conflicting properties will be concatenated with a separator. If two contacts have a conflicting "Note" property. The merged item will have a single "Note" property with the two values concatenated. The delimiter here could be a new line. Available delimiters are space, comma, semi-colon, and new line.
• Drop conflicting properties - With this option, only the value from a master item will be kept during the duplicate items merging. This is useful for technical properties. If your "Contacts" have an "Id", "Created On", or "Last Modified" property, you don't need to combine them. You only want to keep the values from one item. At the moment, the primary item is the one with the most data. In the future, you will be able to define rules to select your primary item.

If you run the auto-merge algorithm and conflicting items remain. Conflicting properties will be listed. Configure the conflicting properties with the "combine" or "drop" options and rerun the auto-merge to finish the merging process.

Download invoices from your account

You don't need to send me an email to get your invoices! Your invoices are now listed in your account. The invoices can be downloaded as PDFs. Check your Billing page to see your invoices.

Quick feedback system

Datablist is in beta. I'm looking for a maximum of feedback to know your use cases, your issues, and your ideas on how to improve the product.

I've been sending emails to users to learn how they use the product but the response rate is very low. In October, I tested another approach with a very simple feedback system directly in the backoffice. It pops up after you export your collection items; after you use the Duplicates Finder; or after using Datablist for some time. And it shows only once a month.

After a few weeks, I've collected a little less than 30 ratings. Mostly good ratings 🤩 and interesting feedback.

Fixes & improvements

• Add an error message when loading an empty CSV
• Show a warning message to free users to alert their data is not synced with the cloud
• Fix check credits balance regularly when running an action on a large collection
• Fix cloud saving on checkbox data
• Fix cloud synchronization after deleting a property
• Fix copy/pasting text in the search input
• Fix auto merging of duplicates with values with different case

Run JavaScript code

Data transformation will be a focus for the next months. Splitting or joining properties, find and replace, etc. They are part of your day-to-day data-cleaning tasks.

I wanted to implement a first dev-friendly feature to run javascript code directly on your collection items. You can clean and transform any of your properties' data by writing a JavaScript function. Check our guide to scraping and enriching Facebook Group members to see how it can be used.

Credit system

Datablist goal is to be the perfect mix of a productivity tool for data management and business software to help you grow your company. Data management is not enough to make an impact. Native data enrichment services and third-party APIs integration will be at the core.

In marketing, SaaS APIs offer email validation, business and people enrichment, scoring, etc. Instead of moving your data from one tool to the next. Datablist will consolidate your data so you trigger each service directly from it.

Every service charges a per-use fee and this cost has to be passed to Datablist customers. The first step toward this vision is a new credit system. Every month, customers receive 5000 credits to be used during the month. And top ups are available to buy extra credits. Free users receive 500 credits on sign-up.

With this system, new third-party integrations will be possible. Feel free to reach me if you want a service to be integrated.

Improvements

Export filtered items

When triggering an export, Datablist will check if you have filters. If your collection is filtered, two options will be available: export only the filtered items, or export the complete collection.

Prevent the browser to load the previous URL on horizontal scroll when a drawer is open

Web browsers have a native implementation with horizontal scrolling to navigate your URL history. Scroll left to load your previous URL, and right to move forward.

This behavior is counterintuitive with Single Page Applications such as Datablist. In the data listing, you have to scroll right and left to see all your properties. Scroll too much and your browser moves you to another page.

It happened to me many times. I open an item in the drawer, I scroll horizontally to check some data, my scroll goes too far, and I left my current page. And the drawer disappears with my data unsaved.

I don't like to overwrite native browser features so I haven't disabled this behavior on all Datablist pages. But it is now disabled when you have the drawer open.

This will prevent most of the data loss when creating a new item or when running an action.

Fixes

• Fix export on collections with more than 500k items
• Fix export on 1 item collection

Managing collection up to 1.5 millions items

Last year, I focused on building the foundations for Datablist. Users management, the data table, and the basics for dealing with data. Until January 2022, Datablist could only import CSV files with 10k rows or less. This is the current limit you find on Airtable, Coda, etc.

For 2022, I wanted Datablist to deal with listings of at least 1 million items. This is a comfortable limit to deal with logs, product data sets, users, and prospect lists. Spreadsheet tools break when dealing with a few hundred of thousand items.

In July I finally unlocked import for CSV files up to 1.5 million rows! (1 million for free users).

Going higher is not on the roadmap.

I can't find business use cases with needs for more than 1.5 million items. Bigger CSV files are for data science and are used for analytics, in read-only mode. Read-only analytics on big CSV files is possible with tools like Microsoft PowerBI and Datablist doesn't have any advantage.

Datablist shines on data consolidation, enrichment from external files or API, and cleaning (deduplication, merging).

Stoppable import process

By allowing big CSV files, the import process can take a few minutes. We want the user to have a reactive experience when using Datablist. We added a "stop import" button to cancel the import before the end.

Improve search and filtering for large collections

The time to process searches and filtering on your data in Datablist is proportional to the number of items. If your collection has 1 million items, it takes one thousand times longer to filter your data than with a 1k items collection.

To scale, Datablist filtering engine stops once it has found enough results to fill your list view. And when you scroll, it resumes the search to find more results.

With this behavior, searching and filtering on hundred of thousand of items feel the same as searching a small dataset.

-----

On top of that, any processing search is canceled when the search and filtering parameters change. When typing in the search box, a search is run any time you stop typing for some time. When you resume typing to add a keyword, the previous search request is canceled.

Persistent item drawer

I've added a persistent url for any item. When you open an item in the drawer, the url changes with the item persistent url.

Returning on this url on a new tab or in another browser will load the collection and open your item directly.

Improvements

See how many items are returned on a search or filter listing

The way Datablist processes data during a search (see above) means the engine doesn't know how many results can be returned on any query. It just stops when it has enough results to show. That is the reason I don't show a "Counter" all the time with the number of matching items. The process of counting how many items match a query is an intensive operation.

But this information is important when managing a dataset.

The total number of items matching a query is now available when using the select all feature.

Toggle the master checkbox, and click on "All items selected". Datablist will count all the results matching your query and replace the text with the value.