Datablist is a <a href="/csv-editor">CSV editor</a> with a built-in character count feature. Many character counters count the characters across all your CSV content. Datablist Character Counter lets you count the character on a specific column. Making it more suitable to real-life use cases.
<h2 id="user-content-how-many-characters-are-in-my-csv-file"><a href="#how-many-characters-are-in-my-csv-file" aria-hidden="true" tabindex="-1"></a>How many characters are in my CSV file?</h2>
First, load your CSV file into Datablist. The Character Count feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load <a href="/how-to/edit-big-csv-files-online">CSV files with up to 1.5 million rows</a>.
Click the "+" button to create a new collection and then click "Import CSV" to load your file.
<img src="/learn_images/csv/characters-count-column/import-csv.png#2155x1326" alt="Import CSV file">
Once loaded, you will see your CSV columns and your CSV content directly in Datablist. Click on the CSV column on which you want to count characters.
<img src="/learn_images/csv/characters-count-column/start-calculations.png#2152x910" alt="Open Calculations tool">
Select "Characters count" in the calculation options. The character count is displayed directly in the drawer.
<img src="/learn_images/csv/characters-count-column/character-count-results.png#2155x1321" alt="Character Count Result">
<h2 id="user-content-how-character-count-is-calculated"><a href="#how-character-count-is-calculated" aria-hidden="true" tabindex="-1"></a>How character count is calculated?</h2>
Every character counts. Spaces, punctuations, digits, smileys, etc. count.
Please note that some smileys are coded on 2 characters. For example, the smiley "🍆" counts as 2 characters.
Line breaks don't count as characters. In the text:
<pre><code>My


Word
</code></pre>
The character count is 6.
<h2 id="user-content-when-to-use-character-count"><a href="#when-to-use-character-count" aria-hidden="true" tabindex="-1"></a>When to use character count?</h2>
Here are some situations where character count is important:
<ul>
<li>SEO Optimization - In the digital age, character count plays a crucial role in SEO. Meta descriptions, titles, and URLs with optimal character lengths are more likely to be indexed effectively by search engines, improving your online visibility.</li>
<li>Social Media - Platforms like "X" (Twitter) have character limits, making it essential to keep tweets within a specific length to convey your message effectively.</li>
<li>Cost Analysis - When using third-party tools, for example, <a href="/how-to/translate-csv-files-online">translation services</a>, where the cost is charged based on word or character count.</li>
</ul>

Datablist is a [CSV editor](/csv-editor) with a built-in character count feature. Many character counters count the characters across all your CSV content. Datablist Character Counter lets you count the character on a specific column. Making it more suitable to real-life use cases.

## How many characters are in my CSV file?

First, load your CSV file into Datablist. The Character Count feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load [CSV files with up to 1.5 million rows](/how-to/edit-big-csv-files-online).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

Click the "+" button to create a new collection and then click "Import CSV" to load your file.

![Import CSV file](/learn_images/csv/characters-count-column/import-csv.png#2155x1326)

Once loaded, you will see your CSV columns and your CSV content directly in Datablist. Click on the CSV column on which you want to count characters.

![Open Calculations tool](/learn_images/csv/characters-count-column/start-calculations.png#2152x910)

Select "Characters count" in the calculation options. The character count is displayed directly in the drawer.

![Character Count Result](/learn_images/csv/characters-count-column/character-count-results.png#2155x1321)

## How character count is calculated?

Every character counts. Spaces, punctuations, digits, smileys, etc. count.

Please note that some smileys are coded on 2 characters. For example, the smiley "🍆" counts as 2 characters.

Line breaks don't count as characters. In the text:

```
My


Word
```

The character count is 6.


## When to use character count?

Here are some situations where character count is important:

- **SEO Optimization** - In the digital age, character count plays a crucial role in SEO. Meta descriptions, titles, and URLs with optimal character lengths are more likely to be indexed effectively by search engines, improving your online visibility.
- **Social Media** - Platforms like "X" (Twitter) have character limits, making it essential to keep tweets within a specific length to convey your message effectively.
- **Cost Analysis** - When using third-party tools, for example, [translation services](/how-to/translate-csv-files-online), where the cost is charged based on word or character count.

Datablist is a CSV editor with a built-in character count feature. Many character counters count the characters across all your CSV content. Datablist Character Counter lets you count the character on a specific column. Making it more suitable to real-life use cases.
How many characters are in my CSV file?
First, load your CSV file into Datablist. The Character Count feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load CSV files with up to 1.5 million rows.
Click the &ampquot;+&ampquot; button to create a new collection and then click &ampquot;Import CSV&ampquot; to load your file.
Import CSV file
Once loaded, you will see your CSV columns and your CSV content directly in Datablist. Click on the CSV column on which you want to count characters.
Open Calculations tool
Select &ampquot;Characters count&ampquot; in the calculation options. The character count is displayed directly in the drawer.
Character Count Result
How character count is calculated?
Every character counts. Spaces, punctuations, digits, smileys, etc. count.
Please note that some smileys are coded on 2 characters. For example, the smiley &ampquot;🍆&ampquot; counts as 2 characters.
Line breaks don&amp#39;t count as characters. In the text:
My


Word
The character count is 6.
When to use character count?
Here are some situations where character count is important:
SEO Optimization - In the digital age, character count plays a crucial role in SEO. Meta descriptions, titles, and URLs with optimal character lengths are more likely to be indexed effectively by search engines, improving your online visibility.
Social Media - Platforms like &ampquot;X&ampquot; (Twitter) have character limits, making it essential to keep tweets within a specific length to convey your message effectively.
Cost Analysis - When using third-party tools, for example, translation services, where the cost is charged based on word or character count.



How to count characters in a CSV column

Datablist is a <a href="/csv-editor">CSV editor</a> with a built-in tool to get distinct values from a CSV column. And you get the sum of occurrences for each value.
<img src="/learn_images/csv/distinct-values-column/distinct-values-header.png#1862x1009" alt="Example Distinct Values">
This tutorial shows you the step-by-step process to count and aggregate distinct values with a free online tool. No development or technical skills are required.
<h2 id="user-content-get-distinct-values-in-a-csv-file"><a href="#get-distinct-values-in-a-csv-file" aria-hidden="true" tabindex="-1"></a>Get distinct values in a CSV file</h2>
First, load your CSV file into Datablist. The Distinct Values feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load <a href="/how-to/edit-big-csv-files-online">CSV files with up to 1.5 million rows</a>.
Create a new collection ("+" button in the left sidebar) and then click "Import CSV" to load your file.
<img src="/learn_images/csv/distinct-values-column/import-csv.png#2155x1326" alt="Import CSV file">
CSV columns and your CSV content is visible directly in Datablist in a few seconds. Click on the CSV column you want to analyze.
<img src="/learn_images/csv/distinct-values-column/start-calculations.png#2314x1164" alt="Open Calculations tool">
Select "Count distinct values" in the calculation options. Other options include <a href="/learn/csv/word-count-csv-file">Word Count</a>, and <a href="/learn/csv/characters-count-csv-file">Character Count</a>.
<img src="/learn_images/csv/distinct-values-column/select-calculation-distinct.png#1792x1030" alt="Calculation Selection">
Then run the process. Datablist analyses your data in real-time without sending the data to servers. You get results in a few seconds.
<img src="/learn_images/csv/distinct-values-column/distinct-values-results.png#2320x1202" alt="Distinct Value Results">
<h2 id="user-content-distinct-value-algorithm-explained"><a href="#distinct-value-algorithm-explained" aria-hidden="true" tabindex="-1"></a>Distinct value algorithm explained</h2>
Datablist converts all values to lowercase before doing the aggregation. The algorithm is thus "case insensitive".
Leading and trailing spaces are also removed.
The terms:
<pre><code>PARIS
</code></pre>
and
<pre><code> paRis
</code></pre>
Are similar and will be aggregated.
<h2 id="user-content-when-to-use-the-distinct-values-feature"><a href="#when-to-use-the-distinct-values-feature" aria-hidden="true" tabindex="-1"></a>When to use the distinct values feature?</h2>
This is great for aggregation of limited choice values (countries, status, etc.). For example if your CSV c
<ul>
<li>Leads Analysis - Get insights on your <a href="/use-cases/lead-list-building">prospects list</a> by running the distinct values counter on your prospect outreach status.</li>
<li>Geographical Analytics - Countries, Cities, and other location information can be aggregated to have a better understanding of your dataset.</li>
<li>Data Cleaning - Find outliers and irrelevant data by looking at the values with few occurrences. <a href="/how-to/data-cleaning">Learn more about Data Cleaning</a>.</li>
</ul>

Datablist is a [CSV editor](/csv-editor) with a built-in tool to get distinct values from a CSV column. And you get the sum of occurrences for each value.

![Example Distinct Values](/learn_images/csv/distinct-values-column/distinct-values-header.png#1862x1009)

This tutorial shows you the step-by-step process to count and aggregate distinct values with a free online tool. No development or technical skills are required.

## Get distinct values in a CSV file

First, load your CSV file into Datablist. The Distinct Values feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load [CSV files with up to 1.5 million rows](/how-to/edit-big-csv-files-online).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

Create a new collection ("+" button in the left sidebar) and then click "Import CSV" to load your file.

![Import CSV file](/learn_images/csv/distinct-values-column/import-csv.png#2155x1326)

CSV columns and your CSV content is visible directly in Datablist in a few seconds. Click on the CSV column you want to analyze.

![Open Calculations tool](/learn_images/csv/distinct-values-column/start-calculations.png#2314x1164)

Select "Count distinct values" in the calculation options. Other options include [Word Count](/learn/csv/word-count-csv-file), and [Character Count](/learn/csv/characters-count-csv-file).

![Calculation Selection](/learn_images/csv/distinct-values-column/select-calculation-distinct.png#1792x1030)

Then run the process. Datablist analyses your data in real-time without sending the data to servers. You get results in a few seconds.

![Distinct Value Results](/learn_images/csv/distinct-values-column/distinct-values-results.png#2320x1202)

## Distinct value algorithm explained

Datablist converts all values to lowercase before doing the aggregation. The algorithm is thus "case insensitive".
Leading and trailing spaces are also removed.

The terms:

```
PARIS
```

and

```
 paRis
```

Are similar and will be aggregated.


## When to use the distinct values feature?

This is great for aggregation of limited choice values (countries, status, etc.). For example if your CSV c

- **Leads Analysis** - Get insights on your [prospects list](/use-cases/lead-list-building) by running the distinct values counter on your prospect outreach status.
- **Geographical Analytics** - Countries, Cities, and other location information can be aggregated to have a better understanding of your dataset.
- **Data Cleaning** - Find outliers and irrelevant data by looking at the values with few occurrences. [Learn more about Data Cleaning](/how-to/data-cleaning).

Datablist is a CSV editor with a built-in tool to get distinct values from a CSV column. And you get the sum of occurrences for each value.
Example Distinct Values
This tutorial shows you the step-by-step process to count and aggregate distinct values with a free online tool. No development or technical skills are required.
Get distinct values in a CSV file
First, load your CSV file into Datablist. The Distinct Values feature is available without registration. You can import a CSV file with a maximum of 10000 rows as an anonymous. For larger CSV files, just create an account for free. Registered users can load CSV files with up to 1.5 million rows.
Create a new collection (&ampquot;+&ampquot; button in the left sidebar) and then click &ampquot;Import CSV&ampquot; to load your file.
Import CSV file
CSV columns and your CSV content is visible directly in Datablist in a few seconds. Click on the CSV column you want to analyze.
Open Calculations tool
Select &ampquot;Count distinct values&ampquot; in the calculation options. Other options include Word Count, and Character Count.
Calculation Selection
Then run the process. Datablist analyses your data in real-time without sending the data to servers. You get results in a few seconds.
Distinct Value Results
Distinct value algorithm explained
Datablist converts all values to lowercase before doing the aggregation. The algorithm is thus &ampquot;case insensitive&ampquot;.
Leading and trailing spaces are also removed.
The terms:
PARIS
and
      paRis
Are similar and will be aggregated.
When to use the distinct values feature?
This is great for aggregation of limited choice values (countries, status, etc.). For example if your CSV c
Leads Analysis - Get insights on your prospects list by running the distinct values counter on your prospect outreach status.
Geographical Analytics - Countries, Cities, and other location information can be aggregated to have a better understanding of your dataset.
Data Cleaning - Find outliers and irrelevant data by looking at the values with few occurrences. Learn more about Data Cleaning.



How to count distinct values in a CSV column?

Comma Separated Value(s), also known as CSV, is a format to store structured data using text files.
The CSV format is not standardized, so various implementations exist.
In 2005, <a href="https://www.internetsociety.org/" target="_blank" rel="nofollow">the Internet Society</a> published <a href="https://www.ietf.org/rfc/rfc4180.txt" target="_blank" rel="nofollow">guidelines for creating CSV files</a>. They wrote down best practices to structure and process CSV data.
From those guidelines and giving the lack of standardization, the header line is optional in a CSV file.
When present, the header line must be the first line in the file and must contain the same number of fields as the records. Header and records lines must use the same field delimiters.
Example:
<pre><code>column_1, column2, column3
aaa, bbb, ccc
111, 222, 333
</code></pre>


Comma Separated Value(s), also known as CSV, is a format to store structured data using text files.
**The CSV format is not standardized, so various implementations exist.**

In 2005, [the Internet Society](https://www.internetsociety.org/) published [guidelines for creating CSV files](https://www.ietf.org/rfc/rfc4180.txt). They wrote down best practices to structure and process CSV data.

From those guidelines and giving the lack of standardization, the header line is **optional** in a CSV file.

When present, the header line must be the first line in the file and must contain the same number of fields as the records. Header and records lines must use the same field delimiters.

Example:

```
column_1, column2, column3
aaa, bbb, ccc
111, 222, 333
```


Comma Separated Value(s), also known as CSV, is a format to store structured data using text files.
The CSV format is not standardized, so various implementations exist.
In 2005, the Internet Society published guidelines for creating CSV files. They wrote down best practices to structure and process CSV data.
From those guidelines and giving the lack of standardization, the header line is optional in a CSV file.
When present, the header line must be the first line in the file and must contain the same number of fields as the records. Header and records lines must use the same field delimiters.
Example:
column_1, column2, column3
aaa, bbb, ccc
111, 222, 333


Do I need to have a header line in my Comma Separated Value file?

CSV stands for "Comma Separated Value(s)" and is a format to store structured data using text files. In a CSV file, each line is a data record. And each record is made of fields separated by commas (or sometimes by semicolons ";" or tabular keys).
CSV files are a common way to transfer data between applications. Because it relies on text files, the CSV format is a simple way to export database or spreadsheet data.
Unfortunately, the CSV file format is not standardized. Encoding format, delimiters, or escaping rules might differ between implementations. As a result, most applications accepting CSV files as data sources have different options to read the CSV data.
To view the content of an CSV file, you can uses an online <a href="/csv-editor">CSV editor</a> that is compatible with CSV exported by most applications.


CSV stands for "Comma Separated Value(s)" and is a format to store structured data using text files. In a CSV file, each line is a data record. And each record is made of fields separated by commas (or sometimes by semicolons ";" or tabular keys).

CSV files are a common way to transfer data between applications. Because it relies on text files, the CSV format is a simple way to export database or spreadsheet data.

Unfortunately, the CSV file format is not standardized. Encoding format, delimiters, or escaping rules might differ between implementations. As a result, most applications accepting CSV files as data sources have different options to read the CSV data.

To view the content of an CSV file, you can uses an online [CSV editor](/csv-editor) that is compatible with CSV exported by most applications.


CSV stands for &ampquot;Comma Separated Value(s)&ampquot; and is a format to store structured data using text files. In a CSV file, each line is a data record. And each record is made of fields separated by commas (or sometimes by semicolons &ampquot;;&ampquot; or tabular keys).
CSV files are a common way to transfer data between applications. Because it relies on text files, the CSV format is a simple way to export database or spreadsheet data.
Unfortunately, the CSV file format is not standardized. Encoding format, delimiters, or escaping rules might differ between implementations. As a result, most applications accepting CSV files as data sources have different options to read the CSV data.
To view the content of an CSV file, you can uses an online CSV editor that is compatible with CSV exported by most applications.


What is a CSV file?

A CSV file is a text file used to store structured data. The CSV format is used to transfer structured data between applications or to export data in order to perform data analytics.
The first use case is data transfer. Applications such as e-commerce software, mailing list providers, or CRM deal can import and export data in the CSV format. In e-commerce software, it is common to import a CSV containing the product catalog. When managing a mailing list, importing a CSV with all the email addresses is faster than adding each contact one by one.
The second use case is data analytics. The CSV format is flat and is perfectly compatible with spreadsheet or data analytics tools. As a result, it is a good source to analyze and visualize data. A CSV file has no size limit and can contain millions or more of records. It is a good file format to use in Big Data.


A CSV file is a text file used to store structured data. The CSV format is used to transfer structured data between applications or to export data in order to perform data analytics.

The first use case is data transfer. Applications such as e-commerce software, mailing list providers, or CRM deal can import and export data in the CSV format. In e-commerce software, it is common to import a CSV containing the product catalog. When managing a mailing list, importing a CSV with all the email addresses is faster than adding each contact one by one.

The second use case is data analytics. The CSV format is flat and is perfectly compatible with spreadsheet or data analytics tools. As a result, it is a good source to analyze and visualize data. A CSV file has no size limit and can contain millions or more of records. It is a good file format to use in Big Data.


A CSV file is a text file used to store structured data. The CSV format is used to transfer structured data between applications or to export data in order to perform data analytics.
The first use case is data transfer. Applications such as e-commerce software, mailing list providers, or CRM deal can import and export data in the CSV format. In e-commerce software, it is common to import a CSV containing the product catalog. When managing a mailing list, importing a CSV with all the email addresses is faster than adding each contact one by one.
The second use case is data analytics. The CSV format is flat and is perfectly compatible with spreadsheet or data analytics tools. As a result, it is a good source to analyze and visualize data. A CSV file has no size limit and can contain millions or more of records. It is a good file format to use in Big Data.


When to use a CSV file?

Dedupe is a synonym for data deduplication or de-duplicate. Dedupe is the process of eliminating duplicate entries from a data set.
The term is used both in computing and for non-technical people.
In computing, deduping is performed in software development to remove duplicates in a list. Still with computing, data deduplication is used with files too. In a system that store files, deduplication is a technique to list all similar files and their reference in documents, emails, etc., and to remove all redundant files to keep only one version and have all references point to those single files.
Dedupe is also used for non-technical people to define the action of finding and merging redundant duplicates. A good example is CRM deduplication. CRM applications store contact information for leads, customers, etc. Several CRM records can represent the same person or business and hold different and complementary data. In this example, <a href="/features/duplicates-remover">merging duplicates</a> in the CRM is important to build a single view for every user. It also lower software cost as it reduces the number of items to store in a database.
<img src="/learn_images/csv/merge-duplicate-rows-modal.png#1732x1060" alt="Merging tool to dedupe contacts">
<a href="/how-to/remove-csv-duplicates">👉 Read our guide to dedupe CSV files</a>


Dedupe is a synonym for **data deduplication** or **de-duplicate**. Dedupe is the process of eliminating duplicate entries from a data set.

The term is used both in **computing** and for **non-technical people**.

**In computing**, deduping is performed in software development to remove duplicates in a list. Still with computing, data deduplication is used with files too. In a system that store files, deduplication is a technique to list all similar files and their reference in documents, emails, etc., and to remove all redundant files to keep only one version and have all references point to those single files.

**Dedupe is also used for non-technical people to define the action of finding and merging redundant duplicates.** A good example is CRM deduplication. CRM applications store contact information for leads, customers, etc. Several CRM records can represent the same person or business and hold different and complementary data. In this example, [merging duplicates](/features/duplicates-remover) in the CRM is important to build a single view for every user. It also lower software cost as it reduces the number of items to store in a database.

![Merging tool to dedupe contacts](/learn_images/csv/merge-duplicate-rows-modal.png#1732x1060)

[👉 Read our guide to dedupe CSV files](/how-to/remove-csv-duplicates)


Dedupe is a synonym for data deduplication or de-duplicate. Dedupe is the process of eliminating duplicate entries from a data set.
The term is used both in computing and for non-technical people.
In computing, deduping is performed in software development to remove duplicates in a list. Still with computing, data deduplication is used with files too. In a system that store files, deduplication is a technique to list all similar files and their reference in documents, emails, etc., and to remove all redundant files to keep only one version and have all references point to those single files.
Dedupe is also used for non-technical people to define the action of finding and merging redundant duplicates. A good example is CRM deduplication. CRM applications store contact information for leads, customers, etc. Several CRM records can represent the same person or business and hold different and complementary data. In this example, merging duplicates in the CRM is important to build a single view for every user. It also lower software cost as it reduces the number of items to store in a database.
Merging tool to dedupe contacts
👉 Read our guide to dedupe CSV files


What tools to dedupe a CSV file?

Join and Merge are two operations to combine data from several files. When merging, you are combining several files with the same structure into a single listing.
When joining, you are combining several files with different data structure but with at least one common field. This common field will be used as a key to combine the data and generate a single listing combining the fields from all the files.
<h2 id="user-content-merge-and-join-examples"><a href="#merge-and-join-examples" aria-hidden="true" tabindex="-1"></a>Merge and Join examples</h2>
<h3 id="user-content-merge-example"><a href="#merge-example" aria-hidden="true" tabindex="-1"></a>Merge example</h3>
Let's say you have a File A:
<pre><code> firstName | lastName | jobTitle
 Jean | Doe | CEO
 Morgan | Stanlet | Marketing Officer
</code></pre>
And a File B:
<pre><code> firstName | lastName | jobTitle
 Joe | Kanigan | Finance
 Marie | Filman | Dev
</code></pre>
Then, the resulting of a merge operation would be:
<pre><code> firstName | lastName | jobTitle
 Jean | Doe | CEO
 Morgan | Stanlet | Marketing Officer
 Joe | Kanigan | Finance
 Marie | Filman | Dev
</code></pre>
<h3 id="user-content-join-example"><a href="#join-example" aria-hidden="true" tabindex="-1"></a>Join example</h3>
Now, let's look at the join operation. When performing a join, your files can have different structures but must have a common identifier that will be used to combine the data together.
We have a File C:
<pre><code> email | jobTitle
 jean.doe@gmail.com | CEO
 morgan@gmail.com | Marketing Officer
</code></pre>
Than we want to join with a File D:
<pre><code> email | City
 jean.doe@gmail.com | Paris
 morgan@gmail.com | New York
</code></pre>
The resulting join operation using the <code>email</code> identifier is
<pre><code> email | jobTitle | City
 jean.doe@gmail.com | CEO | Paris
 morgan@gmail.com | Marketing Officer | New York
</code></pre>
A very common data manipulation task is to bring two or more sets of data together based on a common key.


Join and Merge are two operations to combine data from several files. When merging, you are combining several files **with the same structure** into a single listing.

When joining, you are combining several files **with different data structure** but with at least one common field. This common field will be used as a key to combine the data and generate a single listing combining the fields from all the files.


## Merge and Join examples

### Merge example

Let's say you have a *File A*:

```
  firstName |    lastName    |  jobTitle
    Jean    |       Doe      |  CEO
   Morgan   |     Stanlet    |  Marketing Officer
```


And a *File B*:

```
  firstName |    lastName    |  jobTitle
    Joe     |     Kanigan    |  Finance
   Marie    |     Filman     |  Dev
```

Then, the resulting of a merge operation would be:

```
  firstName |    lastName    |  jobTitle
    Jean    |       Doe      |  CEO
   Morgan   |     Stanlet    |  Marketing Officer
    Joe     |     Kanigan    |  Finance
   Marie    |     Filman     |  Dev
```


### Join example

Now, let's look at the join operation. When performing a join, your files can have different structures but must have a common identifier that will be used to combine the data together.

We have a *File C*:

```
          email           |      jobTitle
    jean.doe@gmail.com    |        CEO
     morgan@gmail.com     |    Marketing Officer
```

Than we want to join with a *File D*:

```
          email           |         City
    jean.doe@gmail.com    |        Paris
     morgan@gmail.com     |       New York
```

The resulting join operation using the `email` identifier is

```
          email           |      jobTitle         |       City
    jean.doe@gmail.com    |        CEO            |       Paris
     morgan@gmail.com     |    Marketing Officer  |      New York
```

A very common data manipulation task is to bring two or more sets of data together based on a common key.


Join and Merge are two operations to combine data from several files. When merging, you are combining several files with the same structure into a single listing.
When joining, you are combining several files with different data structure but with at least one common field. This common field will be used as a key to combine the data and generate a single listing combining the fields from all the files.
Merge and Join examples
Merge example
Let&amp#39;s say you have a File A:
  firstName |    lastName    |  jobTitle
    Jean    |       Doe      |  CEO
   Morgan   |     Stanlet    |  Marketing Officer
And a File B:
  firstName |    lastName    |  jobTitle
    Joe     |     Kanigan    |  Finance
   Marie    |     Filman     |  Dev
Then, the resulting of a merge operation would be:
  firstName |    lastName    |  jobTitle
    Jean    |       Doe      |  CEO
   Morgan   |     Stanlet    |  Marketing Officer
    Joe     |     Kanigan    |  Finance
   Marie    |     Filman     |  Dev
Join example
Now, let&amp#39;s look at the join operation. When performing a join, your files can have different structures but must have a common identifier that will be used to combine the data together.
We have a File C:
          email           |      jobTitle
    jean.doe@gmail.com    |        CEO
     morgan@gmail.com     |    Marketing Officer
Than we want to join with a File D:
          email           |         City
    jean.doe@gmail.com    |        Paris
     morgan@gmail.com     |       New York
The resulting join operation using the email identifier is
          email           |      jobTitle         |       City
    jean.doe@gmail.com    |        CEO            |       Paris
     morgan@gmail.com     |    Marketing Officer  |      New York
A very common data manipulation task is to bring two or more sets of data together based on a common key.


You can download sample CSV files here for testing purposes. The datasets can be used in any software application compatible with CSV files. An easy tool to edit CSV files online is our <a href="/csv-editor">CSV Editor</a>.
Three datasets are available: <a href="#customers-dataset">Customers</a>, <a href="#people-dataset">People</a>, and <a href="#organizations-dataset">Organizations</a>.
For each dataset, several CSV sizes are available, from 100 to 2 million records. The first line contains the CSV headers. An index column is set on each file. Rows have an index value which is incremental and starts at 1 for the first data row.
All datasets are free to download and play with. All the data is random and those files must only be used for testing.
The script used to generate all those CSV files is open source and <a href="https://github.com/datablist/sample-csv-files" target="_blank" rel="nofollow">available on Github</a>.
<h2 id="user-content-customers-dataset"><a href="#customers-dataset" aria-hidden="true" tabindex="-1"></a>Customers Dataset</h2>
<ul>
<li>Index</li>
<li>Customer Id</li>
<li>First Name</li>
<li>Last Name</li>
<li>Company</li>
<li>City</li>
<li>Country</li>
<li>Phone 1</li>
<li>Phone 2</li>
<li>Email</li>
<li>Subscription Date</li>
<li>Website</li>
</ul>
<h3 id="user-content-download-csv-files"><a href="#download-csv-files" aria-hidden="true" tabindex="-1"></a>Download CSV files</h3>
<ul>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.csv" target="_blank" rel="nofollow">organizations-100.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 100 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.csv" target="_blank" rel="nofollow">organizations-1000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 1000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.csv" target="_blank" rel="nofollow">organizations-10000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 10000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.csv" target="_blank" rel="nofollow">organizations-100000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 100000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-500000.zip" target="_blank" rel="nofollow">organizations-500000.csv</a> - Organizations CSV with 500000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000000.zip" target="_blank" rel="nofollow">organizations-1000000.csv</a> - Organizations CSV with 1000000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-2000000.zip" target="_blank" rel="nofollow">organizations-2000000.csv</a> - Organizations CSV with 2000000 records</li>
</ul>
<h2 id="user-content-people-dataset"><a href="#people-dataset" aria-hidden="true" tabindex="-1"></a>People Dataset</h2>
<ul>
<li>Index</li>
<li>User Id</li>
<li>First Name</li>
<li>Last Name</li>
<li>Sex</li>
<li>Email</li>
<li>Phone</li>
<li>Date of birth</li>
<li>Job Title</li>
</ul>
<h3 id="user-content-download-csv-files-1"><a href="#download-csv-files-1" aria-hidden="true" tabindex="-1"></a>Download CSV files</h3>
<ul>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.csv" target="_blank" rel="nofollow">people-100.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.zip" target="_blank" rel="nofollow">Zip version</a> - People CSV with 100 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000.csv" target="_blank" rel="nofollow">people-1000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000.zip" target="_blank" rel="nofollow">Zip version</a> - People CSV with 1000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-10000.csv" target="_blank" rel="nofollow">people-10000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-10000.zip" target="_blank" rel="nofollow">Zip version</a> - People CSV with 10000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100000.csv" target="_blank" rel="nofollow">people-100000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100000.zip" target="_blank" rel="nofollow">Zip version</a> - People CSV with 100000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-500000.zip" target="_blank" rel="nofollow">people-500000.csv</a> - People CSV with 500000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000000.zip" target="_blank" rel="nofollow">people-1000000.csv</a> - People CSV with 1000000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/people/people-2000000.zip" target="_blank" rel="nofollow">people-2000000.csv</a> - People CSV with 2000000 records</li>
</ul>
<h2 id="user-content-organizations-dataset"><a href="#organizations-dataset" aria-hidden="true" tabindex="-1"></a>Organizations Dataset</h2>
<ul>
<li>Index</li>
<li>Organization Id</li>
<li>Name</li>
<li>Website</li>
<li>Country</li>
<li>Description</li>
<li>Founded</li>
<li>Industry</li>
<li>Number of employees</li>
</ul>
<h3 id="user-content-download-csv-files-2"><a href="#download-csv-files-2" aria-hidden="true" tabindex="-1"></a>Download CSV files</h3>
<ul>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.csv" target="_blank" rel="nofollow">organizations-100.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 100 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.csv" target="_blank" rel="nofollow">organizations-1000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 1000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.csv" target="_blank" rel="nofollow">organizations-10000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 10000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.csv" target="_blank" rel="nofollow">organizations-100000.csv</a> - <a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.zip" target="_blank" rel="nofollow">Zip version</a> - Organizations CSV with 100000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-500000.zip" target="_blank" rel="nofollow">organizations-500000.csv</a> - Organizations CSV with 500000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000000.zip" target="_blank" rel="nofollow">organizations-1000000.csv</a> - Organizations CSV with 1000000 records</li>
<li><a href="https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-2000000.zip" target="_blank" rel="nofollow">organizations-2000000.csv</a> - Organizations CSV with 2000000 records</li>
</ul>


You can download sample CSV files here for testing purposes. The datasets can be used in any software application compatible with CSV files. An easy tool to edit CSV files online is our [CSV Editor](/csv-editor).

Three datasets are available: [Customers](#customers-dataset), [People](#people-dataset), and [Organizations](#organizations-dataset).

For each dataset, several CSV sizes are available, from 100 to 2 million records. The first line contains the CSV headers. An index column is set on each file. Rows have an index value which is incremental and starts at 1 for the first data row.

All datasets are **free** to download and play with. All the data is random and those files must only be used for testing.

The script used to generate all those CSV files is open source and [available on Github](https://github.com/datablist/sample-csv-files).


## Customers Dataset

- Index
- Customer Id
- First Name
- Last Name
- Company
- City
- Country
- Phone 1
- Phone 2
- Email
- Subscription Date
- Website

### Download CSV files

- [organizations-100.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.zip) - Organizations CSV with 100 records
- [organizations-1000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.zip) - Organizations CSV with 1000 records
- [organizations-10000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.zip) - Organizations CSV with 10000 records
- [organizations-100000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.zip) - Organizations CSV with 100000 records
- [organizations-500000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-500000.zip) - Organizations CSV with 500000 records
- [organizations-1000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000000.zip) - Organizations CSV with 1000000 records
- [organizations-2000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-2000000.zip) - Organizations CSV with 2000000 records


## People Dataset

- Index
- User Id
- First Name
- Last Name
- Sex
- Email
- Phone
- Date of birth
- Job Title

### Download CSV files

- [people-100.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100.zip) - People CSV with 100 records
- [people-1000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000.zip) - People CSV with 1000 records
- [people-10000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-10000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-10000.zip) - People CSV with 10000 records
- [people-100000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-100000.zip) - People CSV with 100000 records
- [people-500000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-500000.zip) - People CSV with 500000 records
- [people-1000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-1000000.zip) - People CSV with 1000000 records
- [people-2000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/people/people-2000000.zip) - People CSV with 2000000 records


## Organizations Dataset

- Index
- Organization Id
- Name
- Website
- Country
- Description
- Founded
- Industry
- Number of employees

### Download CSV files

- [organizations-100.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100.zip) - Organizations CSV with 100 records
- [organizations-1000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000.zip) - Organizations CSV with 1000 records
- [organizations-10000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-10000.zip) - Organizations CSV with 10000 records
- [organizations-100000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.csv) - [Zip version](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-100000.zip) - Organizations CSV with 100000 records
- [organizations-500000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-500000.zip) - Organizations CSV with 500000 records
- [organizations-1000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-1000000.zip) - Organizations CSV with 1000000 records
- [organizations-2000000.csv](https://github.com/datablist/sample-csv-files/raw/main/files/organizations/organizations-2000000.zip) - Organizations CSV with 2000000 records


You can download sample CSV files here for testing purposes. The datasets can be used in any software application compatible with CSV files. An easy tool to edit CSV files online is our CSV Editor.
Three datasets are available: Customers, People, and Organizations.
For each dataset, several CSV sizes are available, from 100 to 2 million records. The first line contains the CSV headers. An index column is set on each file. Rows have an index value which is incremental and starts at 1 for the first data row.
All datasets are free to download and play with. All the data is random and those files must only be used for testing.
The script used to generate all those CSV files is open source and available on Github.
Customers Dataset
Index
Customer Id
First Name
Last Name
Company
City
Country
Phone 1
Phone 2
Email
Subscription Date
Website

Download CSV files
organizations-100.csv - Zip version - Organizations CSV with 100 records
organizations-1000.csv - Zip version - Organizations CSV with 1000 records
organizations-10000.csv - Zip version - Organizations CSV with 10000 records
organizations-100000.csv - Zip version - Organizations CSV with 100000 records
organizations-500000.csv - Organizations CSV with 500000 records
organizations-1000000.csv - Organizations CSV with 1000000 records
organizations-2000000.csv - Organizations CSV with 2000000 records

People Dataset
Index
User Id
First Name
Last Name
Sex
Email
Phone
Date of birth
Job Title

Download CSV files
people-100.csv - Zip version - People CSV with 100 records
people-1000.csv - Zip version - People CSV with 1000 records
people-10000.csv - Zip version - People CSV with 10000 records
people-100000.csv - Zip version - People CSV with 100000 records
people-500000.csv - People CSV with 500000 records
people-1000000.csv - People CSV with 1000000 records
people-2000000.csv - People CSV with 2000000 records

Organizations Dataset
Index
Organization Id
Name
Website
Country
Description
Founded
Industry
Number of employees

Download CSV files
organizations-100.csv - Zip version - Organizations CSV with 100 records
organizations-1000.csv - Zip version - Organizations CSV with 1000 records
organizations-10000.csv - Zip version - Organizations CSV with 10000 records
organizations-100000.csv - Zip version - Organizations CSV with 100000 records
organizations-500000.csv - Organizations CSV with 500000 records
organizations-1000000.csv - Organizations CSV with 1000000 records
organizations-2000000.csv - Organizations CSV with 2000000 records



<ol>
<li>Open the Terminal. (on macOs, you can use Spotlight with keyboard shortcut ⌘ + Space)
<img src="/learn_images/csv/words-count/open-terminal.png#1675x607" alt="Open Terminal using Mac Spotlight"></li>
<li>Move terminal working directory to the folder containing your CSV file. Use ls to show current directory files and folders and cd to move inside a folder.</li>
<li>Enter the following command. Replace filepath with your CSV file path:</li>
</ol>
<pre><code>wc -l filepath.csv
</code></pre>
<ol start="4">
<li>The line count will be printed before the file name
<img src="/learn_images/csv/words-count/word-count.png#1448x518" alt="Word Count Command"></li>
</ol>
The wc command is used to count the number of words or lines (with the -l option) in a file. In a CSV file, each line is a data record. Counting the number of rows is an easy way to have the number of records in your CSV file.
<h2 id="user-content-get-the-number-of-rows-containing-a-keyword-in-a-csv-file"><a href="#get-the-number-of-rows-containing-a-keyword-in-a-csv-file" aria-hidden="true" tabindex="-1"></a>Get the number of rows containing a keyword in a CSV file</h2>
If you want to filter rows in your CSV file and count the matching lines, combine wc with the command grep.
<pre><code>grep "keyword" filepath.csv | wc -l
</code></pre>
<img src="/learn_images/csv/words-count/matching-rows-count.png#1336x518" alt="Filter and Count Lines">
You can also search for multiple keywords with grep using a regular expression (the -E option is for --extended-regexp). Any line with at least one of the keywords will be counted.
<pre><code>grep -E "keyword1|keyword2" filepath.csv | wc -l
</code></pre>
<img src="/learn_images/csv/words-count/sum-rows-multiple-patterns.png#1336x462" alt="Filter and Count Lines with multiple keywords">
By default, grep is case sensitive (uppercase and lowercase characters are distinct). To ignore case in the search, add the -i (for --ignore-case) in your command:
<pre><code>grep -i "keyword" filepath.csv | wc -l
</code></pre>
or
<pre><code>grep -i -E "keyword1|keyword2" filepath.csv | wc -l
</code></pre>
<h2 id="user-content-get-the-number-of-rows-of-multiple-csv-files"><a href="#get-the-number-of-rows-of-multiple-csv-files" aria-hidden="true" tabindex="-1"></a>Get the number of rows of multiple CSV files</h2>
To have the sum of the rows of all the CSV files in the current folder, use the following command:
<pre><code>cat *.csv | wc -l
</code></pre>
<img src="/learn_images/csv/words-count/sum-rows-csv-files.png#1336x518" alt="Get the sum of all csv files">
To view the content of your CSV file, you can uses an online <a href="/csv-editor">CSV editor</a>.



 1. Open the Terminal. (on macOs, you can use Spotlight with keyboard shortcut **⌘** + **Space**)
    ![Open Terminal using Mac Spotlight](/learn_images/csv/words-count/open-terminal.png#1675x607)
 2. Move terminal working directory to the folder containing your CSV file. Use **ls** to show current directory files and folders and **cd** to move inside a folder.
 3. Enter the following command. Replace *filepath* with your CSV file path:
 ```
 wc -l filepath.csv
 ```
 4. The line count will be printed before the file name
 ![Word Count Command](/learn_images/csv/words-count/word-count.png#1448x518)


 The **wc** command is used to count the number of words or lines (with the **-l** option) in a file. In a CSV file, each line is a data record. Counting the number of rows is an easy way to have the number of records in your CSV file.


## Get the number of rows containing a keyword in a CSV file

If you want to filter rows in your CSV file and count the matching lines, combine **wc** with the command **grep**.

```
grep "keyword" filepath.csv | wc -l
```

![Filter and Count Lines](/learn_images/csv/words-count/matching-rows-count.png#1336x518)

You can also search for multiple keywords with **grep** using a regular expression (the **-E** option is for **--extended-regexp**). Any line with at least one of the keywords will be counted.

```
grep -E "keyword1|keyword2" filepath.csv | wc -l
```

![Filter and Count Lines with multiple keywords](/learn_images/csv/words-count/sum-rows-multiple-patterns.png#1336x462)

By default, **grep** is case sensitive (uppercase and lowercase characters are distinct). To ignore case in the search, add the **-i** (for --ignore-case) in your command:

```
grep -i "keyword" filepath.csv | wc -l
```
or
```
grep -i -E "keyword1|keyword2" filepath.csv | wc -l
```


## Get the number of rows of multiple CSV files

To have the sum of the rows of all the CSV files in the current folder, use the following command:

```
cat *.csv | wc -l
```

![Get the sum of all csv files](/learn_images/csv/words-count/sum-rows-csv-files.png#1336x518)



To view the content of your CSV file, you can uses an online [CSV editor](/csv-editor).


Open the Terminal. (on macOs, you can use Spotlight with keyboard shortcut ⌘ + Space)
Open Terminal using Mac Spotlight
Move terminal working directory to the folder containing your CSV file. Use ls to show current directory files and folders and cd to move inside a folder.
Enter the following command. Replace filepath with your CSV file path:wc -l filepath.csv

The line count will be printed before the file name
Word Count Command

 The wc command is used to count the number of words or lines (with the -l option) in a file. In a CSV file, each line is a data record. Counting the number of rows is an easy way to have the number of records in your CSV file.
Get the number of rows containing a keyword in a CSV file
If you want to filter rows in your CSV file and count the matching lines, combine wc with the command grep.
grep &quotkeyword&quot filepath.csv | wc -l
Filter and Count Lines
You can also search for multiple keywords with grep using a regular expression (the -E option is for --extended-regexp). Any line with at least one of the keywords will be counted.
grep -E &quotkeyword1|keyword2&quot filepath.csv | wc -l
Filter and Count Lines with multiple keywords
By default, grep is case sensitive (uppercase and lowercase characters are distinct). To ignore case in the search, add the -i (for --ignore-case) in your command:
grep -i &quotkeyword&quot filepath.csv | wc -l
or
grep -i -E &quotkeyword1|keyword2&quot filepath.csv | wc -l
Get the number of rows of multiple CSV files
To have the sum of the rows of all the CSV files in the current folder, use the following command:
cat *.csv | wc -l
Get the sum of all csv files
To view the content of your CSV file, you can uses an online CSV editor.


Count the number of rows of a CSV file on Linux or macOS Terminal

CSV files are simple text files. A way to look at it is to use a text editor. But CSV files contain structured data and can be hard to navigate with a text editor.
The easier way to open a CSV file is using a spreadsheet application.
Microsoft Excel, Apple Numbers, or Google Sheets can open <code>.csv</code> files. Once open, you will be able to view and edit your data directly in the spreadsheet application.
<img src="/learn_images/csv/google_sheet_csv.png#1616x886" alt="CSV file in Google Sheets">
However, spreadsheet apps are designed for accounting and lack common features when it comes to structured data management. In <a href="https://www.datablist.com/" target="_blank" rel="nofollow">Datablist</a>, CSV files are first-class citizens! Open, define data types, and manage your data directly in our <a href="https://www.datablist.com/csv-editor" target="_blank" rel="nofollow">CSV Editor</a>.
Larger files (more than 1 million rows) require specific tools. For those files, use data analytics software, big data tools, or custom development to process your files.


CSV files are simple text files. A way to look at it is to use a text editor. But CSV files contain structured data and can be hard to navigate with a text editor.

The easier way to open a CSV file is using a spreadsheet application.
Microsoft Excel, Apple Numbers, or Google Sheets can open `.csv` files. Once open, you will be able to view and edit your data directly in the spreadsheet application.

![CSV file in Google Sheets](/learn_images/csv/google_sheet_csv.png#1616x886)

However, spreadsheet apps are designed for accounting and lack common features when it comes to structured data management. In [Datablist](https://www.datablist.com/), CSV files are first-class citizens! Open, define data types, and manage your data directly in our [CSV Editor](https://www.datablist.com/csv-editor).

**Larger files (more than 1 million rows) require specific tools.** For those files, use data analytics software, big data tools, or custom development to process your files.


CSV files are simple text files. A way to look at it is to use a text editor. But CSV files contain structured data and can be hard to navigate with a text editor.
The easier way to open a CSV file is using a spreadsheet application.
Microsoft Excel, Apple Numbers, or Google Sheets can open .csv files. Once open, you will be able to view and edit your data directly in the spreadsheet application.
CSV file in Google Sheets
However, spreadsheet apps are designed for accounting and lack common features when it comes to structured data management. In Datablist, CSV files are first-class citizens! Open, define data types, and manage your data directly in our CSV Editor.
Larger files (more than 1 million rows) require specific tools. For those files, use data analytics software, big data tools, or custom development to process your files.


What tools to open a CSV file?

Swapping and/or removing CSV columns is common in data manipulation.
<a href="/csv-editor">Datablist CSV Editor</a> is the easiest way to reorder and/or remove columns in a CSV file online. For free.
<h2 id="user-content-step-1-open-your-csv-online"><a href="#step-1-open-your-csv-online" aria-hidden="true" tabindex="-1"></a>Step 1: Open your CSV online</h2>
First, open your CSV file online with Datablist <a href="/csv-editor">CSV Editor</a>.
Then, click on the "Start with a CSV/Excel file" button.
<img src="/learn_images/csv/reorder-csv-columns/open-csv-file.png#2145x1320" alt="Open CSV file">
Drag and drop your CSV file, or click on the grey zone to browse your files.
<img src="/learn_images/csv/reorder-csv-columns/load-csv-file.png#2147x1319" alt="Load CSV file">
Follow the instructions. Check your data in the preview step. If you see incorrect characters, try changing the encoding options in the first step.
If the preview data is good, click "Import" to import your CSV data.
<img src="/learn_images/csv/reorder-csv-columns/preview-csv-file.png#2436x1399" alt="CSV Preview">
<h2 id="user-content-step-2-swap-andor-remove-csv-columns"><a href="#step-2-swap-andor-remove-csv-columns" aria-hidden="true" tabindex="-1"></a>Step 2: Swap and/or remove CSV columns</h2>
Datablist is a <a href="/csv-editor">CSV Editor</a>. Once your CSV is imported, you can see your CSV data in the data table. Edit your data: cell by cell or with bulk edit; search items; or perform <a href="/how-to/data-cleaning">data cleaning manipulations</a>.
CSV columns are called properties within Datablist. Click on the "Manage Properties" button to reorder and/or hide CSV columns.
<img src="/learn_images/csv/reorder-csv-columns/manage-properties.png#2433x1392" alt="Properties Configuration">
<ul>
<li>Drag and drop the properties by clicking on the left icon to rearrange them.</li>
<li>Toggle the visibility button on the right to hide a property.</li>
</ul>
<img src="/learn_images/csv/reorder-csv-columns/reorder-hide-csv-columns.png#2437x1411" alt="ReOrder-Remove CSV columns">
<img src="/learn_images/csv/reorder-csv-columns/reordering-csv-columns.png#1869x1107" alt="ReOrdering CSV column">
<h2 id="user-content-step-3-export-the-cleaned-csv-data"><a href="#step-3-export-the-cleaned-csv-data" aria-hidden="true" tabindex="-1"></a>Step 3: Export the cleaned CSV data</h2>
When your manipulations are finished. Click the "Export" button and select "Export only visible properties".
A CSV file will be generated with columns in the same order as displayed.
<img src="/learn_images/csv/reorder-csv-columns/export-cleaned-csv.png#2164x1164" alt="Export CSV">
<h2 id="user-content-faq"><a href="#faq" aria-hidden="true" tabindex="-1"></a>FAQ</h2>
<h3 id="user-content-do-i-need-to-be-a-developer-to-reorder-andor-remove-csv-columns"><a href="#do-i-need-to-be-a-developer-to-reorder-andor-remove-csv-columns" aria-hidden="true" tabindex="-1"></a>Do I need to be a developer to reorder and/or remove CSV columns?</h3>
No. Datablist is a no-code online tool for everyone!
<h3 id="user-content-how-to-export-in-a-microsoft-excel-file"><a href="#how-to-export-in-a-microsoft-excel-file" aria-hidden="true" tabindex="-1"></a>How to export in a Microsoft Excel file?</h3>
Datablist is compatible with Microsoft Excel format. Select "Microsoft Excel" in the export modal to generate an Excel file from your CSV data.

Swapping and/or removing CSV columns is common in data manipulation.

**[Datablist CSV Editor](/csv-editor) is the easiest way to reorder and/or remove columns in a CSV file online. For free.**

## Step 1: Open your CSV online

First, open your CSV file online with Datablist [CSV Editor](/csv-editor).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

Then, click on the "Start with a CSV/Excel file" button.

![Open CSV file](/learn_images/csv/reorder-csv-columns/open-csv-file.png#2145x1320)

Drag and drop your CSV file, or click on the grey zone to browse your files.

![Load CSV file](/learn_images/csv/reorder-csv-columns/load-csv-file.png#2147x1319)

Follow the instructions. Check your data in the preview step. If you see incorrect characters, try changing the encoding options in the first step.

**If the preview data is good, click "Import" to import your CSV data.**

![CSV Preview](/learn_images/csv/reorder-csv-columns/preview-csv-file.png#2436x1399)


## Step 2: Swap and/or remove CSV columns

Datablist is a [CSV Editor](/csv-editor). Once your CSV is imported, you can see your CSV data in the data table. Edit your data: cell by cell or with bulk edit; search items; or perform [data cleaning manipulations](/how-to/data-cleaning).

CSV columns are called properties within Datablist. **Click on the "Manage Properties" button to reorder and/or hide CSV columns.**

![Properties Configuration](/learn_images/csv/reorder-csv-columns/manage-properties.png#2433x1392)

- Drag and drop the properties by clicking on the left icon to rearrange them.
- Toggle the visibility button on the right to hide a property.

![ReOrder-Remove CSV columns](/learn_images/csv/reorder-csv-columns/reorder-hide-csv-columns.png#2437x1411)

![ReOrdering CSV column](/learn_images/csv/reorder-csv-columns/reordering-csv-columns.png#1869x1107)



## Step 3: Export the cleaned CSV data

When your manipulations are finished. Click the "Export" button and select "Export only visible properties".

A CSV file will be generated with columns in the same order as displayed.

![Export CSV](/learn_images/csv/reorder-csv-columns/export-cleaned-csv.png#2164x1164)


## FAQ

### Do I need to be a developer to reorder and/or remove CSV columns?

No. Datablist is a no-code online tool for everyone!

### How to export in a Microsoft Excel file?

Datablist is compatible with Microsoft Excel format. Select "Microsoft Excel" in the export modal to generate an Excel file from your CSV data.

Swapping and/or removing CSV columns is common in data manipulation.
Datablist CSV Editor is the easiest way to reorder and/or remove columns in a CSV file online. For free.
Step 1: Open your CSV online
First, open your CSV file online with Datablist CSV Editor.
Then, click on the &ampquot;Start with a CSV/Excel file&ampquot; button.
Open CSV file
Drag and drop your CSV file, or click on the grey zone to browse your files.
Load CSV file
Follow the instructions. Check your data in the preview step. If you see incorrect characters, try changing the encoding options in the first step.
If the preview data is good, click &ampquot;Import&ampquot; to import your CSV data.
CSV Preview
Step 2: Swap and/or remove CSV columns
Datablist is a CSV Editor. Once your CSV is imported, you can see your CSV data in the data table. Edit your data: cell by cell or with bulk edit; search items; or perform data cleaning manipulations.
CSV columns are called properties within Datablist. Click on the &ampquot;Manage Properties&ampquot; button to reorder and/or hide CSV columns.
Properties Configuration
Drag and drop the properties by clicking on the left icon to rearrange them.
Toggle the visibility button on the right to hide a property.

ReOrder-Remove CSV columns
ReOrdering CSV column
Step 3: Export the cleaned CSV data
When your manipulations are finished. Click the &ampquot;Export&ampquot; button and select &ampquot;Export only visible properties&ampquot;.
A CSV file will be generated with columns in the same order as displayed.
Export CSV
FAQ
Do I need to be a developer to reorder and/or remove CSV columns?
No. Datablist is a no-code online tool for everyone!
How to export in a Microsoft Excel file?
Datablist is compatible with Microsoft Excel format. Select &ampquot;Microsoft Excel&ampquot; in the export modal to generate an Excel file from your CSV data.


How to reorder and/or remove CSV columns online?

Few <a href="/csv-editor">CSV editors</a> can manage large CSV files. Spreadsheet tools like Excel and Google Sheet break before reaching the million of lines. And dedicated tools target data scientists so they are expensive and complex to use.
A free alternative is to use your terminal and manage your big CSV file using the command line.
A CSV file is a text file, with each line being a data record. And each record is made of fields separated by commas (or sometimes by semicolons ";" or tabular keys). Because a CSV file is a text file and each line is a record, you can use Linux/Unix commands for manipulating texts.
The command line might seem scary but they are also very powerful. I'm listed here the command I use to search in big CSV files.
<blockquote>
Note: All the commands listed bellow are just reading your files. They don't overwrite or delete your CSV files.
</blockquote>
A summary of the commands listed on this page:
<ul>
<li><a href="#searching-in-a-big-csv-file-for-a">Searching in a CSV file for a single keyword</a></li>
<li><a href="#searching-in-a-big-csv-file-for-multiple-keywords">Searching in a CSV file for multiple keywords</a></li>
<li><a href="#how-to-count-the-number-of-matching-rows">How to count the number of matching rows in a big CSV file</a></li>
</ul>
Before moving to the commands, open your Terminal. On macOs, you can use Spotlight with keyboard shortcut ⌘ + Space and type Terminal. Then hit "Enter"/"Return" to open the Terminal.
<img src="/learn_images/csv/words-count/open-terminal.png#1675x607" alt="Open Terminal using Mac Spotlight">
<h2 id="user-content-searching-in-a-big-csv-file-for-a-single-keyword"><a href="#searching-in-a-big-csv-file-for-a-single-keyword" aria-hidden="true" tabindex="-1"></a>Searching in a big CSV file for a single keyword</h2>
To perform a simple search and list all rows that match a specific keyword, use <a href="#grep">grep</a>.
<pre><code>grep "keyword" filepath.csv
</code></pre>
To be case insensitive, add the -i option.
<pre><code>grep -i "keyword" filepath.csv
</code></pre>
<img src="/learn_images/csv/search-big-csv-file-terminal/simple-search-case-insensitive.png#916x406=.with-shadow.rounded-3.mb-4" alt="Search CSV for a single keyword">
In my example, I search in a companies dataset the lines matching the keyword airtable.com. I get the two matching rows. One for the company with the website airtable.com (the good one), and another for chinaairtable.com (unexpected).
<h4 id="user-content-search-for-whole-keyword"><a href="#search-for-whole-keyword" aria-hidden="true" tabindex="-1"></a>Search for whole keyword</h4>
To avoid unexpected results like that, you can switch to Regular Expression (RegEx) and search for the whole keyword.
By adding the -E option to grep, I tell grep to interpret the pattern as a regular expression. And I add \b before and after my keyword. \b is a RegEx operator that means word boundary.
<pre><code>grep -E -i "\bkeyword\b" filepath.csv
</code></pre>
<img src="/learn_images/csv/search-big-csv-file-terminal/simple-search-whole-word.png#986x322=.with-shadow.rounded-3.mb-4" alt="Search CSV for a single keyword with word boundary">
<h4 id="user-content-search-terms-starting-with-a-keyword"><a href="#search-terms-starting-with-a-keyword" aria-hidden="true" tabindex="-1"></a>Search terms starting with a keyword</h4>
By using the same RegEx operator \b at the beginning of your pattern, you are searching all the lines with a word starting with your keyword.
<pre><code>grep -E -i "\bkeyword" filepath.csv
</code></pre>
<h4 id="user-content-search-terms-ending-with-a-keyword"><a href="#search-terms-ending-with-a-keyword" aria-hidden="true" tabindex="-1"></a>Search terms ending with a keyword</h4>
And by using it at the end of your pattern, you are searching the lines with a word ending with your keyword.
<pre><code>grep -E -i "keyword\b" filepath.csv
</code></pre>
<img src="/learn_images/csv/search-big-csv-file-terminal/simple-search-ending-width.png#1476x1106=.with-shadow.rounded-3.mb-4" alt="Search CSV for words ending with a keyword">
<h2 id="user-content-searching-in-a-big-csv-file-for-multiple-keywords"><a href="#searching-in-a-big-csv-file-for-multiple-keywords" aria-hidden="true" tabindex="-1"></a>Searching in a big CSV file for multiple keywords</h2>
When searching for multiple keywords in a CSV file, you can search for one of the keywords or for all the keywords.
<h3 id="user-content-searching-for-one-of-several-keywords-in-a-big-csv-file"><a href="#searching-for-one-of-several-keywords-in-a-big-csv-file" aria-hidden="true" tabindex="-1"></a>Searching for one of several keywords in a big CSV file</h3>
Grep with a RegEx is once again the perfect choice. In a RegEx, the symbol | means OR.
<pre><code>grep -i -E "keyword1|keyword2" filepath.csv
</code></pre>
And to search for whole keywords, we can add \b for each keyword.
<pre><code>grep -i -E "\bkeyword1\b|\bkeyword2\b" filepath.csv
</code></pre>
<img src="/learn_images/csv/search-big-csv-file-terminal/multiple-keywords-or.png#1252x434=.with-shadow.rounded-3.mb-4" alt="Search CSV for multiple keywords">
<h3 id="user-content-searching-for-all-the-keywords-in-a-big-csv-file"><a href="#searching-for-all-the-keywords-in-a-big-csv-file" aria-hidden="true" tabindex="-1"></a>Searching for all the keywords in a big CSV file</h3>
To search for lines in your CSV file that have all the requested keywords, you can use another command: awk. Awk is another command tool to deal with text files. It has more advanced features.
Awk also works with RegEx and interprets text enclosed in slashes (/) as regular expressions.
<pre><code>awk '/keyword1/ &#x26;&#x26; /keyword2/' filepath.csv
</code></pre>
Awk processes the file line by line. To be case insensitive, you can convert the line to lower text before searching your keywords:
<pre><code>awk '{lc=tolower($0)}; (lc ~ /keyword1/) &#x26;&#x26; (lc ~ /keyword2/)' filepath.csv
</code></pre>
Here, we take the current line $0 and create a variable lc containing the current line in lower case. Then we check for keywords in it.
In the screenshot below, I search in my CSV for a line with the keywords france AND google.
<img src="/learn_images/csv/search-big-csv-file-terminal/exact-multiple-keywords.png#1378x938=.with-shadow.rounded-3.mb-4" alt="Search CSV for all keywords">
<h2 id="user-content-how-to-count-the-number-of-matching-rows"><a href="#how-to-count-the-number-of-matching-rows" aria-hidden="true" tabindex="-1"></a>How to count the number of matching rows?</h2>
Several commands can be combined. An important command is wc for word count. It does a great job at counting words or lines in a text file.
Use the -l option to count lines in a CSV file or after another command.
For example, to count how many lines match a keyword, combine it with the <a href="#searching-in-a-big-csv-file-for-a-single-keyword">single keyword search</a>.
<pre><code>grep -i "keyword" filepath.csv | wc -l
</code></pre>
<img src="/learn_images/csv/search-big-csv-file-terminal/search-and-count.png#1028x350=.with-shadow.rounded-3.mb-4" alt="Count how many lines match a keyword">


Few [CSV editors](/csv-editor) can manage large CSV files. Spreadsheet tools like Excel and Google Sheet break before reaching the million of lines. And dedicated tools target data scientists so they are expensive and complex to use.

**A free alternative** is to use your terminal and manage your big CSV file using the command line.

A CSV file is a text file, with each line being a data record. And each record is made of fields separated by commas (or sometimes by semicolons ";" or tabular keys). Because a CSV file is a text file and each line is a record, you can use Linux/Unix commands for manipulating texts.

The command line might seem scary but they are also very powerful. I'm listed here the command I use to search in big CSV files.

> **Note**: All the commands listed bellow are just **reading** your files. They don't overwrite or delete your CSV files.

A summary of the commands listed on this page:

- [Searching in a CSV file for a single keyword](#searching-in-a-big-csv-file-for-a)
- [Searching in a CSV file for multiple keywords](#searching-in-a-big-csv-file-for-multiple-keywords)
- [How to count the number of matching rows in a big CSV file](#how-to-count-the-number-of-matching-rows)


Before moving to the commands, open your Terminal. On macOs, you can use Spotlight with keyboard shortcut **⌘** + **Space** and type **Terminal**. Then hit "Enter"/"Return" to open the Terminal.

![Open Terminal using Mac Spotlight](/learn_images/csv/words-count/open-terminal.png#1675x607)


## Searching in a big CSV file for a single keyword


To perform a simple search and list all rows that match a specific keyword, use [grep](#grep).

```
grep "keyword" filepath.csv
```

To be case insensitive, add the **-i** option.

```
grep -i "keyword" filepath.csv
```

![Search CSV for a single keyword](/learn_images/csv/search-big-csv-file-terminal/simple-search-case-insensitive.png#916x406=.with-shadow.rounded-3.mb-4)


In my example, I search in a companies dataset the lines matching the keyword **airtable.com**. I get the two matching rows. One for the company with the website **airtable.com** (the good one), and another for **chinaairtable.com** (unexpected).

#### Search for whole keyword

To avoid unexpected results like that, you can switch to Regular Expression (RegEx) and search for the whole keyword.

By adding the **-E** option to **grep**, I tell grep to interpret the pattern as a regular expression. And I add **\b** before and after my keyword. **\b** is a RegEx operator that means word boundary.

```
grep -E -i "\bkeyword\b" filepath.csv
```
![Search CSV for a single keyword with word boundary](/learn_images/csv/search-big-csv-file-terminal/simple-search-whole-word.png#986x322=.with-shadow.rounded-3.mb-4)


#### Search terms starting with a keyword

By using the same RegEx operator **\b** at the beginning of your pattern, you are searching all the lines with a word starting with your keyword.

```
grep -E -i "\bkeyword" filepath.csv
```

#### Search terms ending with a keyword

And by using it at the end of your pattern, you are searching the lines with a word ending with your keyword.

```
grep -E -i "keyword\b" filepath.csv
```

![Search CSV for words ending with a keyword](/learn_images/csv/search-big-csv-file-terminal/simple-search-ending-width.png#1476x1106=.with-shadow.rounded-3.mb-4)



## Searching in a big CSV file for multiple keywords

When searching for multiple keywords in a CSV file, you can search for **one of** the keywords or for **all** the keywords.

### Searching for one of several keywords in a big CSV file

Grep with a RegEx is once again the perfect choice. In a RegEx, the symbol **|** means OR.

```
grep -i -E "keyword1|keyword2" filepath.csv
```

And to search for whole keywords, we can add **\b** for each keyword.

```
grep -i -E "\bkeyword1\b|\bkeyword2\b" filepath.csv
```

![Search CSV for multiple keywords](/learn_images/csv/search-big-csv-file-terminal/multiple-keywords-or.png#1252x434=.with-shadow.rounded-3.mb-4)


### Searching for all the keywords in a big CSV file

To search for lines in your CSV file that have all the requested keywords, you can use another command: **awk**. Awk is another command tool to deal with text files. It has more advanced features.

Awk also works with RegEx and interprets text enclosed in slashes (**/**) as regular expressions.

```
awk '/keyword1/ && /keyword2/' filepath.csv
```

Awk processes the file line by line. To be case insensitive, you can convert the line to lower text before searching your keywords:

```
awk '{lc=tolower($0)}; (lc ~ /keyword1/) && (lc ~ /keyword2/)' filepath.csv
```

Here, we take the current line **$0** and create a variable **lc** containing the current line in lower case. Then we check for keywords in it.

In the screenshot below, I search in my CSV for a line with the keywords **france** AND **google**.

![Search CSV for all keywords](/learn_images/csv/search-big-csv-file-terminal/exact-multiple-keywords.png#1378x938=.with-shadow.rounded-3.mb-4)


## How to count the number of matching rows?

Several commands can be combined. An important command is **wc** for **word count**. It does a great job at counting words or lines in a text file.

Use the **-l** option to count lines in a CSV file or after another command.

For example, to count how many lines match a keyword, combine it with the [single keyword search](#searching-in-a-big-csv-file-for-a-single-keyword).


```
grep -i "keyword" filepath.csv | wc -l
```

![Count how many lines match a keyword](/learn_images/csv/search-big-csv-file-terminal/search-and-count.png#1028x350=.with-shadow.rounded-3.mb-4)


Few CSV editors can manage large CSV files. Spreadsheet tools like Excel and Google Sheet break before reaching the million of lines. And dedicated tools target data scientists so they are expensive and complex to use.
A free alternative is to use your terminal and manage your big CSV file using the command line.
A CSV file is a text file, with each line being a data record. And each record is made of fields separated by commas (or sometimes by semicolons &ampquot;;&ampquot; or tabular keys). Because a CSV file is a text file and each line is a record, you can use Linux/Unix commands for manipulating texts.
The command line might seem scary but they are also very powerful. I&amp#39;m listed here the command I use to search in big CSV files.
Note: All the commands listed bellow are just reading your files. They don&amp#39;t overwrite or delete your CSV files.

A summary of the commands listed on this page:
Searching in a CSV file for a single keyword
Searching in a CSV file for multiple keywords
How to count the number of matching rows in a big CSV file

Before moving to the commands, open your Terminal. On macOs, you can use Spotlight with keyboard shortcut ⌘ + Space and type Terminal. Then hit &ampquot;Enter&ampquot;/&ampquot;Return&ampquot; to open the Terminal.
Open Terminal using Mac Spotlight
Searching in a big CSV file for a single keyword
To perform a simple search and list all rows that match a specific keyword, use grep.
grep &quotkeyword&quot filepath.csv
To be case insensitive, add the -i option.
grep -i &quotkeyword&quot filepath.csv
Search CSV for a single keyword
In my example, I search in a companies dataset the lines matching the keyword airtable.com. I get the two matching rows. One for the company with the website airtable.com (the good one), and another for chinaairtable.com (unexpected).
Search for whole keyword
To avoid unexpected results like that, you can switch to Regular Expression (RegEx) and search for the whole keyword.
By adding the -E option to grep, I tell grep to interpret the pattern as a regular expression. And I add \b before and after my keyword. \b is a RegEx operator that means word boundary.
grep -E -i &quot\bkeyword\b&quot filepath.csv
Search CSV for a single keyword with word boundary
Search terms starting with a keyword
By using the same RegEx operator \b at the beginning of your pattern, you are searching all the lines with a word starting with your keyword.
grep -E -i &quot\bkeyword&quot filepath.csv
Search terms ending with a keyword
And by using it at the end of your pattern, you are searching the lines with a word ending with your keyword.
grep -E -i &quotkeyword\b&quot filepath.csv
Search CSV for words ending with a keyword
Searching in a big CSV file for multiple keywords
When searching for multiple keywords in a CSV file, you can search for one of the keywords or for all the keywords.
Searching for one of several keywords in a big CSV file
Grep with a RegEx is once again the perfect choice. In a RegEx, the symbol | means OR.
grep -i -E &quotkeyword1|keyword2&quot filepath.csv
And to search for whole keywords, we can add \b for each keyword.
grep -i -E &quot\bkeyword1\b|\bkeyword2\b&quot filepath.csv
Search CSV for multiple keywords
Searching for all the keywords in a big CSV file
To search for lines in your CSV file that have all the requested keywords, you can use another command: awk. Awk is another command tool to deal with text files. It has more advanced features.
Awk also works with RegEx and interprets text enclosed in slashes (/) as regular expressions.
awk &#39/keyword1/ &amp&amp /keyword2/&#39 filepath.csv
Awk processes the file line by line. To be case insensitive, you can convert the line to lower text before searching your keywords:
awk &#39{lc=tolower($0)}; (lc ~ /keyword1/) &amp&amp (lc ~ /keyword2/)&#39 filepath.csv
Here, we take the current line $0 and create a variable lc containing the current line in lower case. Then we check for keywords in it.
In the screenshot below, I search in my CSV for a line with the keywords france AND google.
Search CSV for all keywords
How to count the number of matching rows?
Several commands can be combined. An important command is wc for word count. It does a great job at counting words or lines in a text file.
Use the -l option to count lines in a CSV file or after another command.
For example, to count how many lines match a keyword, combine it with the single keyword search.
grep -i &quotkeyword&quot filepath.csv | wc -l
Count how many lines match a keyword


Search big CSV file using Linux or macOS Terminal

You have a very big CSV file, and you want to split it into several files of n lines. Smaller files let you open them with <a href="/csv-editor">CSV editors</a> without reaching their limits.
If you are not afraid of using the terminal, macOS and Linux come with a simple utility to split files into parts: <a href="https://man7.org/linux/man-pages/man1/split.1.html" target="_blank" rel="nofollow">split</a>. <a href="https://man7.org/linux/man-pages/man1/split.1.html" target="_blank" rel="nofollow">See full documentation on split man page.</a>
<pre><code>split [-l line_count] [-b byte_count] [-n chunk_count] [file [prefix]]
</code></pre>
The command <code>split</code> breaks up files using:
<ul>
<li>lines - useful for text-based files</li>
<li>bytes - for binary files</li>
<li>number - if you want to split a file into a specific number of parts.</li>
</ul>
<a href="/learn/csv/csv-meaning">CSV is a format to store structured data using text files.</a> In a CSV file, each line is a data record. To split a CSV file, we'll use <code>split</code> with the <code>lines</code> parameter.
<pre><code>split -l 100000 filename.csv
</code></pre>
<img src="/learn_images/csv/split-csv-file/basic-split-file.png#1392x350=.with-shadow.rounded-3.mb-4" alt="Basic split CSV file">
<code>Split</code> generates files with basic names beginning with x: <code>xaa</code>, <code>xab</code>, <code>xac</code>, etc. See the next section to split your CSV files and rename the generated files.
<h3 id="user-content-using-split-and-renaming-generated-files"><a href="#using-split-and-renaming-generated-files" aria-hidden="true" tabindex="-1"></a>Using split and renaming generated files</h3>
<code>Split</code> has two interesting parameters. The first is -d to use numeric suffixes (1, 2, 3, etc.) instead of letters. The second is the prefix parameter for the generated files. To define the prefix, just add a string after your CSV filename.
<pre><code>split -l 100000 -d filename.csv file_
</code></pre>
<img src="/learn_images/csv/split-csv-file/split-numeric-prefix.png#1182x350=.with-shadow.rounded-3.mb-4" alt="split CSV file with prefix and numeric suffixes">
Then, loop on all files starting with <code>file_</code> and rename them to add <code>.csv</code>.
<pre><code>for i in $(find file_*); do mv $i "$i.csv"; done
</code></pre>
<img src="/learn_images/csv/split-csv-file/split-prefix-rename.png#944x518=.with-shadow.rounded-3.mb-4" alt="Renaming generated files with .csv">
<h3 id="user-content-adding-csv-header-to-generated-files"><a href="#adding-csv-header-to-generated-files" aria-hidden="true" tabindex="-1"></a>Adding CSV header to generated files</h3>
You have several generated files like: <code>file_00.csv</code>, <code>file_01.csv</code>, etc.
But the <a href="/learn/csv/csv-headers">CSV header line</a> is on the first file (<code>file_00.csv</code>) and all other files don't have a CSV header line.
To copy the file_00.csv first line on all other files, run the following command:
<pre><code>for i in $(find . -type f -name "file_*.csv" -not -name "file_00.csv");
 do echo -e "$(head -1 file_00.csv)\n$(cat $i)" > $i;
done
</code></pre>
What it does is simple:
Step1
Loop on all file that begin with file_ but are not file_00.csv:
<code>find . -type f -name "file_*.csv" -not -name "file_00.csv"</code>
Step2
For each file in the loop, copy the first line of <code>file_00.csv</code> to it. There is no easy way to prepend a line to a file using the command line. We just use <code>echo</code> with the first line from file_00.csv and the file content from the current file <code>cat $i</code>.
<h3 id="user-content-to-sum-up-the-whole-process"><a href="#to-sum-up-the-whole-process" aria-hidden="true" tabindex="-1"></a>To Sum up the whole process</h3>
Step 1: Split the CSV file into parts of 100000 lines and prefix the generated files with "file_".
<pre><code>split -l 100000 -d filename.csv file_
</code></pre>
Step 2: Add <code>.csv</code> to all generated files.
<pre><code>for i in $(find file_*); do mv $i "$i.csv"; done
</code></pre>
Step 3: Copy header from first generated file at the beginning of the other files.
<pre><code>for i in $(find . -type f -name "file_*.csv" -not -name "file_00.csv");
 do echo -e "$(head -1 file_00.csv)\n$(cat $i)" > $i;
done
</code></pre>


You have a very big CSV file, and you want to split it into several files of n lines. Smaller files let you open them with [CSV editors](/csv-editor) without reaching their limits.

If you are not afraid of using the terminal, macOS and Linux come with a simple utility to split files into parts: [**split**](https://man7.org/linux/man-pages/man1/split.1.html). [See full documentation on split man page.](https://man7.org/linux/man-pages/man1/split.1.html)

```
split [-l line_count] [-b byte_count] [-n chunk_count] [file [prefix]]
```

The command `split` breaks up files using:
- **lines** - useful for text-based files
- **bytes** - for binary files
- **number** - if you want to split a file into a specific number of parts.

[CSV is a format to store structured data using text files.](/learn/csv/csv-meaning) In a CSV file, each line is a data record. **To split a CSV file, we'll use `split` with the `lines` parameter.**

```
split -l 100000 filename.csv
```

![Basic split CSV file](/learn_images/csv/split-csv-file/basic-split-file.png#1392x350=.with-shadow.rounded-3.mb-4)

`Split` generates files with basic names beginning with **x**: `xaa`, `xab`, `xac`, etc. See the next section to split your CSV files and rename the generated files.

### Using split and renaming generated files

`Split` has two interesting parameters. The first is **-d** to use numeric suffixes (1, 2, 3, etc.) instead of letters. The second is the **prefix** parameter for the generated files. To define the **prefix**, just add a string after your CSV filename.

```
split -l 100000 -d filename.csv file_
```

![split CSV file with prefix and numeric suffixes](/learn_images/csv/split-csv-file/split-numeric-prefix.png#1182x350=.with-shadow.rounded-3.mb-4)

Then, loop on all files starting with `file_` and rename them to add `.csv`.

```
for i in $(find file_*); do mv $i "$i.csv"; done
```

![Renaming generated files with .csv](/learn_images/csv/split-csv-file/split-prefix-rename.png#944x518=.with-shadow.rounded-3.mb-4)


### Adding CSV header to generated files

You have several generated files like: `file_00.csv`, `file_01.csv`, etc.

But the [CSV header line](/learn/csv/csv-headers) is on the first file (`file_00.csv`) and all other files don't have a CSV header line.

To copy the file_00.csv first line on all other files, run the following command:

```
for i in $(find . -type f -name "file_*.csv" -not -name "file_00.csv");
    do echo -e "$(head -1 file_00.csv)\n$(cat $i)" > $i;
done
```

What it does is simple:

**Step1**

Loop on all file that begin with **file_** but are not **file_00.csv**:

`find . -type f -name "file_*.csv" -not -name "file_00.csv"`


**Step2**

For each file in the loop, copy the first line of `file_00.csv` to it. There is no easy way to prepend a line to a file using the command line. We just use `echo` with the first line from file_00.csv and the file content from the current file `cat $i`.


### To Sum up the whole process

Step 1: Split the CSV file into parts of 100000 lines and prefix the generated files with "file_".

```
split -l 100000 -d filename.csv file_
```

Step 2: Add `.csv` to all generated files.

```
for i in $(find file_*); do mv $i "$i.csv"; done
```

Step 3: Copy header from first generated file at the beginning of the other files.
```
for i in $(find . -type f -name "file_*.csv" -not -name "file_00.csv");
    do echo -e "$(head -1 file_00.csv)\n$(cat $i)" > $i;
done
```


You have a very big CSV file, and you want to split it into several files of n lines. Smaller files let you open them with CSV editors without reaching their limits.
If you are not afraid of using the terminal, macOS and Linux come with a simple utility to split files into parts: split. See full documentation on split man page.
split [-l line_count] [-b byte_count] [-n chunk_count] [file [prefix]]
The command split breaks up files using:
lines - useful for text-based files
bytes - for binary files
number - if you want to split a file into a specific number of parts.

CSV is a format to store structured data using text files. In a CSV file, each line is a data record. To split a CSV file, we&amp#39;ll use split with the lines parameter.
split -l 100000 filename.csv
Basic split CSV file
Split generates files with basic names beginning with x: xaa, xab, xac, etc. See the next section to split your CSV files and rename the generated files.
Using split and renaming generated files
Split has two interesting parameters. The first is -d to use numeric suffixes (1, 2, 3, etc.) instead of letters. The second is the prefix parameter for the generated files. To define the prefix, just add a string after your CSV filename.
split -l 100000 -d filename.csv file_
split CSV file with prefix and numeric suffixes
Then, loop on all files starting with file_ and rename them to add .csv.
for i in $(find file_*); do mv $i &quot$i.csv&quot; done
Renaming generated files with .csv
Adding CSV header to generated files
You have several generated files like: file_00.csv, file_01.csv, etc.
But the CSV header line is on the first file (file_00.csv) and all other files don&amp#39;t have a CSV header line.
To copy the file_00.csv first line on all other files, run the following command:
for i in $(find . -type f -name &quotfile_*.csv&quot -not -name &quotfile_00.csv&quot);
    do echo -e &quot$(head -1 file_00.csv)\n$(cat $i)&quot &gt $i;
done
What it does is simple:
Step1
Loop on all file that begin with file_ but are not file_00.csv:
find . -type f -name &ampquot;file_*.csv&ampquot; -not -name &ampquot;file_00.csv&ampquot;
Step2
For each file in the loop, copy the first line of file_00.csv to it. There is no easy way to prepend a line to a file using the command line. We just use echo with the first line from file_00.csv and the file content from the current file cat $i.
To Sum up the whole process
Step 1: Split the CSV file into parts of 100000 lines and prefix the generated files with &ampquot;file_&ampquot;.
split -l 100000 -d filename.csv file_
Step 2: Add .csv to all generated files.
for i in $(find file_*); do mv $i &quot$i.csv&quot; done
Step 3: Copy header from first generated file at the beginning of the other files.
for i in $(find . -type f -name &quotfile_*.csv&quot -not -name &quotfile_00.csv&quot);
    do echo -e &quot$(head -1 file_00.csv)\n$(cat $i)&quot &gt $i;
done


Split big CSV file using Linux or Mac OS Terminal

Datablist is a CSV editor and provides calculation features. Calculations include <a href="/learn/csv/word-count-csv-file">word count</a>, <a href="/learn/csv/characters-count-csv-file">character count</a>, empty-filled counts, etc.
For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.
<h2 id="user-content-how-to-get-the-sum-min-max-and-average-values-of-a-csv-column"><a href="#how-to-get-the-sum-min-max-and-average-values-of-a-csv-column" aria-hidden="true" tabindex="-1"></a>How to get the Sum, Min, Max, and Average values of a CSV column?</h2>
Begin by loading your CSV file in Datablist. Anonymous users (without signup) can load files with up to 10,000 rows. For larger files, just <a href="https://app.datablist.com/auth/signup" target="_blank" rel="nofollow">sign up and create a free account</a>.
<img src="/learn_images/csv/sum-min-max-average-csv-column/import-csv.png#2155x1326" alt="Import CSV file">
Choose the specific column for which you want to perform numeric calculations. Then click on "Perform calculation".
<img src="/learn_images/csv/sum-min-max-average-csv-column/start-calculations.png#2152x910" alt="Open Calculation tool">
Select the desired calculation operation – whether it's sum, minimum, maximum, or average.
<img src="/learn_images/csv/sum-min-max-average-csv-column/min-max-average-calculation.png#2156x1322" alt="Select numeric calculation">
Watch as our feature swiftly processes your data and presents you with the calculated result. No more waiting around or navigating through complex formulas.
<img src="/learn_images/csv/sum-min-max-average-csv-column/min-max-average-calculation-results.png#2156x1322" alt="Get the Sum, Min, Max, and Average value">
<h2 id="user-content-when-to-use-numeric-calculations-on-a-csv-file"><a href="#when-to-use-numeric-calculations-on-a-csv-file" aria-hidden="true" tabindex="-1"></a>When to use Numeric Calculations on a CSV file?</h2>
<ul>
<li>Data Analysts - For fast analytical processes without the need to use a complex data analytic tool.</li>
<li>Traffic Analysis - Get export from the Google Search Console and perform calculations to get aggregated results.</li>
<li>E-commerce Product Catalog analysis - Easily play with numeric values such as prices, order counts, etc. in your e-commerce</li>
</ul>

Datablist is a CSV editor and provides calculation features. Calculations include [word count](/learn/csv/word-count-csv-file), [character count](/learn/csv/characters-count-csv-file), empty-filled counts, etc.

For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.

## How to get the Sum, Min, Max, and Average values of a CSV column?

Begin by loading your CSV file in Datablist. Anonymous users (without signup) can load files with up to 10,000 rows. For larger files, just [sign up and create a free account](https://app.datablist.com/auth/signup).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

![Import CSV file](/learn_images/csv/sum-min-max-average-csv-column/import-csv.png#2155x1326)

Choose the specific column for which you want to perform numeric calculations. Then click on "Perform calculation".

![Open Calculation tool](/learn_images/csv/sum-min-max-average-csv-column/start-calculations.png#2152x910)

Select the desired calculation operation – whether it's sum, minimum, maximum, or average.

![Select numeric calculation](/learn_images/csv/sum-min-max-average-csv-column/min-max-average-calculation.png#2156x1322)

Watch as our feature swiftly processes your data and presents you with the calculated result. No more waiting around or navigating through complex formulas.

![Get the Sum, Min, Max, and Average value](/learn_images/csv/sum-min-max-average-csv-column/min-max-average-calculation-results.png#2156x1322)

## When to use Numeric Calculations on a CSV file?

- **Data Analysts** - For fast analytical processes without the need to use a complex data analytic tool.
- **Traffic Analysis** - Get export from the Google Search Console and perform calculations to get aggregated results.
- **E-commerce Product Catalog analysis** - Easily play with numeric values such as prices, order counts, etc. in your e-commerce

Datablist is a CSV editor and provides calculation features. Calculations include word count, character count, empty-filled counts, etc.
For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.
How to get the Sum, Min, Max, and Average values of a CSV column?
Begin by loading your CSV file in Datablist. Anonymous users (without signup) can load files with up to 10,000 rows. For larger files, just sign up and create a free account.
Import CSV file
Choose the specific column for which you want to perform numeric calculations. Then click on &ampquot;Perform calculation&ampquot;.
Open Calculation tool
Select the desired calculation operation – whether it&amp#39;s sum, minimum, maximum, or average.
Select numeric calculation
Watch as our feature swiftly processes your data and presents you with the calculated result. No more waiting around or navigating through complex formulas.
Get the Sum, Min, Max, and Average value
When to use Numeric Calculations on a CSV file?
Data Analysts - For fast analytical processes without the need to use a complex data analytic tool.
Traffic Analysis - Get export from the Google Search Console and perform calculations to get aggregated results.
E-commerce Product Catalog analysis - Easily play with numeric values such as prices, order counts, etc. in your e-commerce 



Get the Sum, Min, Max, and Average values of a CSV column

Datablist is a <a href="/csv-editor">CSV editor</a> and provides calculation features. Calculations provide word counts, character counts, empty-filled counts, etc.
For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.
<h2 id="user-content-how-many-words-are-in-my-csv-file"><a href="#how-many-words-are-in-my-csv-file" aria-hidden="true" tabindex="-1"></a>How many words are in my CSV file?</h2>
The first step to get the word count is to load your CSV file. Datablist is a powerful <a href="/csv-editor">CSV viewer</a> that loads <a href="/how-to/edit-big-csv-files-online">CSV files with up to 1.5 million rows</a>.
In the <a href="https://app.datablist.com/" target="_blank" rel="nofollow">Datablist Application</a>, create a new collection (using the "+" in the sidebar) to load your CSV file.
<img src="/learn_images/csv/words-count-column/import-csv.png#2155x1326" alt="Import CSV file">
Follow the import wizard to get your CSV data in Datablist. Once the import is done, click on the column you want to count words in. Select "Perform calculation".
<img src="/learn_images/csv/words-count-column/start-calculations.png#2152x910" alt="Open Calculations tool">
The "Calculation" tool opens on the right with the column name pre-filled. Datablist provides several calculations:
<ul>
<li>Words Count</li>
<li><a href="/learn/csv/characters-count-csv-file">Characters Count</a></li>
<li><a href="/learn/csv/sum-min-max-average-csv-column">Min, Max, Average for numeric columns</a></li>
</ul>
<img src="/learn_images/csv/words-count-column/words-count-column.png#2156x1322" alt="Word Count result">
<h2 id="user-content-how-word-count-is-calculated"><a href="#how-word-count-is-calculated" aria-hidden="true" tabindex="-1"></a>How word count is calculated?</h2>
Datablist Word Count analyzes the text of each column's cell. For each, it splits the text on any whitespace character. The word count is the sum of the split parts.
For compound words, such as "rule-breaking", the word count is 1.
Punctuation characters are excluded from the word count. In the French sentence (In French, we have a space before punctuation letters) "Quel est ton nom ?", the word count is 4 ("quel", "est", "ton", "nom").
Numbers are counted as words. In the text "My phone number is 04 38 58 19 38.", the word count is 9.
Spaces before or after a text are ignored, and multi-space characters are ignored. In the text " Datablist is great ", the word count is 3 ("datablist", "is", "great").
<h2 id="user-content-when-to-use-word-count"><a href="#when-to-use-word-count" aria-hidden="true" tabindex="-1"></a>When to use word count?</h2>
Here are some situations where word count is important:
<ul>
<li>Blogging and Online Content - Online content including blog posts and articles, may benefit from maintaining an optimal word count for search engine optimization (SEO). Search engines often favor longer, comprehensive content.</li>
<li>Academic Writing and Professional Writing. - Many academic assignments and professional documents come with specific word count requirements.</li>
<li>Social Media - Platforms like "X" (Twitter) have character limits, making it essential to keep tweets within a specific word count to convey your message effectively.</li>
<li>Cost Analysis - When using third-party tools, for example, <a href="/how-to/translate-csv-files-online">translation services</a>, where the cost is charged based on word or character count.</li>
</ul>

Datablist is a [CSV editor](/csv-editor) and provides calculation features. Calculations provide word counts, character counts, empty-filled counts, etc.

For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.

## How many words are in my CSV file?

The first step to get the word count is to load your CSV file. Datablist is a powerful [CSV viewer](/csv-editor) that loads [CSV files with up to 1.5 million rows](/how-to/edit-big-csv-files-online).

<aside class="text-center mb-5">
 <div class="mb-2">
 <a href="https://app.datablist.com/" className="btn btn-primary btn-large" target="_blank">
 Open Datablist
 </a>
 </div>
 No signup required
</aside>

In the [Datablist Application](https://app.datablist.com/), create a new collection (using the "+" in the sidebar) to load your CSV file.

![Import CSV file](/learn_images/csv/words-count-column/import-csv.png#2155x1326)

Follow the import wizard to get your CSV data in Datablist. Once the import is done, click on the column you want to count words in. Select "Perform calculation".

![Open Calculations tool](/learn_images/csv/words-count-column/start-calculations.png#2152x910)

The "Calculation" tool opens on the right with the column name pre-filled. Datablist provides several calculations:

- Words Count
- [Characters Count](/learn/csv/characters-count-csv-file)
- [Min, Max, Average for numeric columns](/learn/csv/sum-min-max-average-csv-column)

![Word Count result](/learn_images/csv/words-count-column/words-count-column.png#2156x1322)

## How word count is calculated?

Datablist Word Count analyzes the text of each column's cell. For each, it splits the text on any whitespace character. The word count is the sum of the split parts.

For compound words, such as "rule-breaking", the word count is 1.

Punctuation characters are excluded from the word count. In the French sentence (In French, we have a space before punctuation letters) "Quel est ton nom ?", the word count is 4 ("quel", "est", "ton", "nom").

Numbers are counted as words. In the text "My phone number is 04 38 58 19 38.", the word count is 9.

Spaces before or after a text are ignored, and multi-space characters are ignored. In the text " Datablist is great ", the word count is 3 ("datablist", "is", "great").

## When to use word count?

Here are some situations where word count is important:

- **Blogging and Online Content** - Online content including blog posts and articles, may benefit from maintaining an optimal word count for search engine optimization (SEO). Search engines often favor longer, comprehensive content.
- **Academic Writing and Professional Writing.** - Many academic assignments and professional documents come with specific word count requirements.
- **Social Media** - Platforms like "X" (Twitter) have character limits, making it essential to keep tweets within a specific word count to convey your message effectively.
- **Cost Analysis** - When using third-party tools, for example, [translation services](/how-to/translate-csv-files-online), where the cost is charged based on word or character count.

Datablist is a CSV editor and provides calculation features. Calculations provide word counts, character counts, empty-filled counts, etc.
For CSV columns with numeric values, calculations return Sum, Average, Min, and Max.
How many words are in my CSV file?
The first step to get the word count is to load your CSV file. Datablist is a powerful CSV viewer that loads CSV files with up to 1.5 million rows.
In the Datablist Application, create a new collection (using the &ampquot;+&ampquot; in the sidebar) to load your CSV file.
Import CSV file
Follow the import wizard to get your CSV data in Datablist. Once the import is done, click on the column you want to count words in. Select &ampquot;Perform calculation&ampquot;.
Open Calculations tool
The &ampquot;Calculation&ampquot; tool opens on the right with the column name pre-filled. Datablist provides several calculations:
Words Count
Characters Count
Min, Max, Average for numeric columns

Word Count result
How word count is calculated?
Datablist Word Count analyzes the text of each column&amp#39;s cell. For each, it splits the text on any whitespace character. The word count is the sum of the split parts.
For compound words, such as &ampquot;rule-breaking&ampquot;, the word count is 1.
Punctuation characters are excluded from the word count. In the French sentence (In French, we have a space before punctuation letters) &ampquot;Quel est ton nom ?&ampquot;, the word count is 4 (&ampquot;quel&ampquot;, &ampquot;est&ampquot;, &ampquot;ton&ampquot;, &ampquot;nom&ampquot;).
Numbers are counted as words. In the text &ampquot;My phone number is 04 38 58 19 38.&ampquot;, the word count is 9.
Spaces before or after a text are ignored, and multi-space characters are ignored. In the text &ampquot;     Datablist        is great    &ampquot;, the word count is 3 (&ampquot;datablist&ampquot;, &ampquot;is&ampquot;, &ampquot;great&ampquot;).
When to use word count?
Here are some situations where word count is important:
Blogging and Online Content - Online content including blog posts and articles, may benefit from maintaining an optimal word count for search engine optimization (SEO). Search engines often favor longer, comprehensive content.
Academic Writing and Professional Writing. - Many academic assignments and professional documents come with specific word count requirements.
Social Media - Platforms like &ampquot;X&ampquot; (Twitter) have character limits, making it essential to keep tweets within a specific word count to convey your message effectively.
Cost Analysis - When using third-party tools, for example, translation services, where the cost is charged based on word or character count.



How to count words in a CSV Column?

Apple Numbers has a 1,000,000 rows limit.
When importing a big CSV file, only the first million rows are imported. Extra rows are skipped.
<img src="/learn_images/csv/apple-numbers-rows-limit/numbers-import-rows-limits.png#1232x536" alt="Importing a CSV file with more than 1m rows">
This is a hard limit defined in the software. The operational limit is way lower. When editing a 1,000,000 rows spreadsheet with a Macbook Pro M1 16Go, the application freezes for several seconds every time I want to edit a cell.
<img src="/learn_images/csv/apple-numbers-rows-limit/editing-big-csv-numbers.png#2504x1588" alt="Editing big spreadsheet Apple Numbers">
<h2 id="user-content-how-to-get-around-apple-numberss-max-row-limit"><a href="#how-to-get-around-apple-numberss-max-row-limit" aria-hidden="true" tabindex="-1"></a>How to get around Apple Numbers's max row limit?</h2>
Spreadsheet tools are not suited for big spreadsheets. They must keep all the data in memory so users can link any cell in their formulas.
Apple Numbers is not a database and have poor performance on big listing. See our guide on how to <a href="/how-to/edit-big-csv-files-online">edit big CSV files online</a>.


**Apple Numbers has a 1,000,000 rows limit.**

When importing a big CSV file, only the first million rows are imported. Extra rows are skipped.

![Importing a CSV file with more than 1m rows](/learn_images/csv/apple-numbers-rows-limit/numbers-import-rows-limits.png#1232x536)

This is a hard limit defined in the software. The operational limit is way lower. When editing a 1,000,000 rows spreadsheet with a Macbook Pro M1 16Go, the application freezes for several seconds every time I want to edit a cell.

![Editing big spreadsheet Apple Numbers](/learn_images/csv/apple-numbers-rows-limit/editing-big-csv-numbers.png#2504x1588)

## How to get around Apple Numbers's max row limit?

**Spreadsheet tools are not suited for big spreadsheets.** They must keep all the data in memory so users can link any cell in their formulas.
Apple Numbers is not a database and have poor performance on big listing. See our guide on how to [edit big CSV files online](/how-to/edit-big-csv-files-online).


Apple Numbers has a 1,000,000 rows limit.
When importing a big CSV file, only the first million rows are imported. Extra rows are skipped.
Importing a CSV file with more than 1m rows
This is a hard limit defined in the software. The operational limit is way lower. When editing a 1,000,000 rows spreadsheet with a Macbook Pro M1 16Go, the application freezes for several seconds every time I want to edit a cell.
Editing big spreadsheet Apple Numbers
How to get around Apple Numbers&amp#39;s max row limit?
Spreadsheet tools are not suited for big spreadsheets. They must keep all the data in memory so users can link any cell in their formulas.
Apple Numbers is not a database and have poor performance on big listing. See our guide on how to edit big CSV files online.


Apple Numbers maximum rows per spreadsheet

To convert a Numbers (<code>.numbers</code>) file into a <code>.csv</code> file, you can directly use the Apple Numbers app.
First, load your Numbers file file with the Apple Numbers app. Then, use the File -> Export To -> CSV menu action.
<img src="/learn_images/csv/export_apple_numbers_to_csv/apple_numbers_to_csv.png#1488x966" alt="File -> Export To -> CSV">
If needed, change the text encoding format in the "Advanced Options".
<blockquote>
Do not check the "Include table names" option! It adds a first row with the name of the table in your CSV file, which will be considered as a <a href="/learn/csv/csv-headers">CSV header</a> with only one column by most applications.
</blockquote>
<img src="/learn_images/csv/export_apple_numbers_to_csv/apple_numbers_csv_options.png#1410x826" alt="CSV Exporting Options">
Then, click the Next button to select a save location. Finally, click the Export button.


To convert a Numbers (`.numbers`) file into a `.csv` file, you can directly use the Apple Numbers app.

First, load your Numbers file file with the Apple Numbers app. Then, use the **File -> Export To -> CSV** menu action.

![File -> Export To -> CSV](/learn_images/csv/export_apple_numbers_to_csv/apple_numbers_to_csv.png#1488x966)

If needed, change the text encoding format in the "Advanced Options".

> Do not check the "Include table names" option! It adds a first row with the name of the table in your CSV file, which will be considered as a [CSV header](/learn/csv/csv-headers) with only one column by most applications.

![CSV Exporting Options](/learn_images/csv/export_apple_numbers_to_csv/apple_numbers_csv_options.png#1410x826)

Then, click the **Next** button to select a save location. Finally, click the **Export** button.


To convert a Numbers (.numbers) file into a .csv file, you can directly use the Apple Numbers app.
First, load your Numbers file file with the Apple Numbers app. Then, use the File -&ampgt; Export To -&ampgt; CSV menu action.
File -&ampgt; Export To -&ampgt; CSV
If needed, change the text encoding format in the &ampquot;Advanced Options&ampquot;.
Do not check the &ampquot;Include table names&ampquot; option! It adds a first row with the name of the table in your CSV file, which will be considered as a CSV header with only one column by most applications.

CSV Exporting Options
Then, click the Next button to select a save location. Finally, click the Export button.


Export Numbers spreadsheet into a CSV file

Spreadsheet and database both store sets of data. However, each one has advantages and disadvantages and is suited to specific use cases.
A spreadsheet represents a digital worksheet and was invented for accounting tasks. In a spreadsheet, data is stored in a flat structure consisting of rows and columns, also known as tabular format. Spreadsheets are simple to use and require no technical knowledge. Because a spreadsheet is flexible and doesn't have a data schema, any value can be entered in any cell. The drawback of this flexibility is the lack of data schema. Spreadsheets cannot be used safely as a source of data for other applications as they cannot provide a strict structure.
Finally, spreadsheets are designed to handle a limited volume of data.
In databases, data is stored in tables, which have a defined structure. This data structure describes for each table, the attributes and data types expected. For example, attributes can be text, date, text, or relationship to another table. This data structure adds constraints and triggers errors directly when inserting a new record in the database. Thanks to this strict structure, other applications can rely on the data provided by a database.
Unlike spreadsheets, databases are better for storing large amounts of data and when several users or applications access the data simultaneously.


Spreadsheet and database both store sets of data. However, each one has advantages and disadvantages and is suited to specific use cases.

A spreadsheet represents a digital worksheet and was invented for accounting tasks. In a spreadsheet, data is stored in a flat structure consisting of rows and columns, also known as tabular format. Spreadsheets are simple to use and require no technical knowledge. Because a spreadsheet is flexible and doesn't have a data schema, any value can be entered in any cell. The drawback of this flexibility is the lack of data schema. **Spreadsheets cannot be used safely as a source of data for other applications as they cannot provide a strict structure**.
Finally, spreadsheets are designed to handle a limited volume of data.

In databases, data is stored in tables, which have a defined structure. This data structure describes for each table, the attributes and data types expected. For example, attributes can be text, date, text, or relationship to another table. This data structure adds constraints and triggers errors directly when inserting a new record in the database. **Thanks to this strict structure, other applications can rely on the data provided by a database.**
Unlike spreadsheets, databases are better for storing large amounts of data and when several users or applications access the data simultaneously.


Spreadsheet and database both store sets of data. However, each one has advantages and disadvantages and is suited to specific use cases.
A spreadsheet represents a digital worksheet and was invented for accounting tasks. In a spreadsheet, data is stored in a flat structure consisting of rows and columns, also known as tabular format. Spreadsheets are simple to use and require no technical knowledge. Because a spreadsheet is flexible and doesn&amp#39;t have a data schema, any value can be entered in any cell. The drawback of this flexibility is the lack of data schema. Spreadsheets cannot be used safely as a source of data for other applications as they cannot provide a strict structure.
Finally, spreadsheets are designed to handle a limited volume of data.
In databases, data is stored in tables, which have a defined structure. This data structure describes for each table, the attributes and data types expected. For example, attributes can be text, date, text, or relationship to another table. This data structure adds constraints and triggers errors directly when inserting a new record in the database. Thanks to this strict structure, other applications can rely on the data provided by a database.
Unlike spreadsheets, databases are better for storing large amounts of data and when several users or applications access the data simultaneously.


How spreadsheets and databases differ technically?

Exporting a sheet from Microsoft Excel to a CSV file is straightforward. But how to export selected cells to a CSV file?
The short answer is you can't export selected cells in CSV quickly. Exporting a selection to a CSV file or another Excel file is not directly possible. Here is an alternative solution: copy the data to another sheet and export the sheet to a CSV file.
<h2 id="user-content-step-1-copy-paste-selected-cells"><a href="#step-1-copy-paste-selected-cells" aria-hidden="true" tabindex="-1"></a>Step 1: Copy-Paste selected cells</h2>
Select the cells you want to export. And copy the data to your clipboard (<code>ctrl+c</code> or <code>Menu Edit => Copy</code>)
<img src="/learn_images/excel/export-selection-csv/copy-selection.png#1934x1550" alt="File -> Export To -> CSV">
<h3 id="user-content-export-only-visible-cells-without-hidden-columns"><a href="#export-only-visible-cells-without-hidden-columns" aria-hidden="true" tabindex="-1"></a>Export only visible cells (without hidden columns)</h3>
If you have hidden columns, Excel will add their content to your clipboard. This is not what we want. We want to export only our selected and visible cells.
To prevent hidden columns to be copied, open the <code>Go to Special</code> menu in <code>Find &#x26; Select</code>.
<img src="/learn_images/excel/export-selection-csv/go-to-special.png#2676x1552" alt="Find &#x26; Select -> Go to Special">
And select <code>Visible cells only</code>. It will tell Microsoft Excel we only want to copy the visible cells.
<img src="/learn_images/excel/export-selection-csv/visible-cells-only.png#1630x992" alt="Go to special -> Visible cells only">
<h2 id="user-content-step-2-create-a-new-sheet-and-paste-your-data"><a href="#step-2-create-a-new-sheet-and-paste-your-data" aria-hidden="true" tabindex="-1"></a>Step 2: Create a new sheet and paste your data</h2>
Then, create a new sheet and <code>paste</code> your data (<code>ctrl + v</code> or <code>Menu Edit => Paste</code>)
<img src="/learn_images/excel/export-selection-csv/paste-cells.png#2190x1292" alt="Paste data in a new sheet">
<h3 id="user-content-add-columns-to-have-csv-headers"><a href="#add-columns-to-have-csv-headers" aria-hidden="true" tabindex="-1"></a>Add columns to have CSV headers</h3>
<a href="/learn/csv/csv-uses">CSV files are used to store structured data</a>, and CSV headers add context to know what your CSV contains.
<a href="/learn/csv/csv-headers">CSV headers are optional</a>. But if you need to import your CSV into another software or <a href="/csv-editor">clean it with a CSV editor</a>, you better have them.
Just <code>copy - paste</code> the columns from the first sheet or write them directly as the first line in the new sheet.
<img src="/learn_images/excel/export-selection-csv/paste-headers.png#1732x1294" alt="Paste columns in the new sheet">
<h2 id="user-content-step-3-export-sheet-to-csv"><a href="#step-3-export-sheet-to-csv" aria-hidden="true" tabindex="-1"></a>Step 3: Export sheet to CSV</h2>
The last step is to export the new Excel sheet to a CSV file. Once you are on the new sheet, select <code>Save As</code> in the <code>File</code> menu.
<blockquote>
Important: Set the active sheet to the sheet to export!
</blockquote>
<img src="/learn_images/excel/export-selection-csv/save-as-to-export-csv.png#1374x1076" alt="File -> Save As">
Then change the <code>File format</code> to <code>CSV UTF-8 (Comma-delimited) (.csv)</code>. And click <code>Save</code>.
<img src="/learn_images/excel/export-selection-csv/select-csv-as-export-format.png#1780x1294" alt="Save As with CSV format">
A warning message will pop up with the message "This workbook cannot be saved in the selected file format because it contains multiple sheets.".
Bypass the warning message by clicking on <code>OK</code>. Microsoft Excel tells you that it can't export all sheets to a single CSV. Only your active sheet will be exported to a CSV file. This is what we want.
<img src="/learn_images/excel/export-selection-csv/confirm-to-export-active-sheet.png#1512x1076" alt="Confirm with OK">
<h2 id="user-content-edit-it-with-a-csv-editor"><a href="#edit-it-with-a-csv-editor" aria-hidden="true" tabindex="-1"></a>Edit it with a CSV Editor</h2>
If you need to further clean your CSV data, use a <a href="/csv-editor">CSV Editor</a> such as <a href="/csv-editor">Datablist</a>.
Datablist provides advanced tools to <a href="/features/duplicates-remover">deduplicate data</a>, <a href="/features/combine-csv-files">consolidate several CSV files</a>, or <a href="/how-to/clean-web-scraping-data-online">clean your data</a>.


Exporting a sheet from Microsoft Excel to a CSV file is straightforward. But how to export selected cells to a CSV file?

**The short answer is you can't export selected cells in CSV quickly. Exporting a selection to a CSV file or another Excel file is not directly possible. Here is an alternative solution: copy the data to another sheet and export the sheet to a CSV file.** 


## Step 1: Copy-Paste selected cells

Select the cells you want to export. And copy the data to your clipboard (`ctrl+c` or `Menu Edit => Copy`)

![File -> Export To -> CSV](/learn_images/excel/export-selection-csv/copy-selection.png#1934x1550)

### Export only visible cells (without hidden columns)

If you have hidden columns, Excel will add their content to your clipboard. This is not what we want. We want to export only our selected and visible cells.

To prevent hidden columns to be copied, open the `Go to Special` menu in `Find & Select`.


![Find & Select -> Go to Special](/learn_images/excel/export-selection-csv/go-to-special.png#2676x1552)

And select `Visible cells only`. It will tell Microsoft Excel we only want to copy the visible cells.

![Go to special -> Visible cells only](/learn_images/excel/export-selection-csv/visible-cells-only.png#1630x992)


## Step 2: Create a new sheet and paste your data

Then, create a new sheet and `paste` your data (`ctrl + v` or `Menu Edit => Paste`)


![Paste data in a new sheet](/learn_images/excel/export-selection-csv/paste-cells.png#2190x1292)


### Add columns to have CSV headers

[CSV files are used to store structured data](/learn/csv/csv-uses), and CSV headers add context to know what your CSV contains.

[CSV headers are optional](/learn/csv/csv-headers). But if you need to import your CSV into another software or [clean it with a CSV editor](/csv-editor), you better have them.

Just `copy - paste` the columns from the first sheet or write them directly as the first line in the new sheet.

![Paste columns in the new sheet](/learn_images/excel/export-selection-csv/paste-headers.png#1732x1294)

## Step 3: Export sheet to CSV

The last step is to export the new Excel sheet to a CSV file. Once you are on the new sheet, select `Save As` in the `File` menu.

> **Important**: Set the active sheet to the sheet to export!

![File -> Save As](/learn_images/excel/export-selection-csv/save-as-to-export-csv.png#1374x1076)

Then change the `File format` to `CSV UTF-8 (Comma-delimited) (.csv)`. And click `Save`.

![Save As with CSV format](/learn_images/excel/export-selection-csv/select-csv-as-export-format.png#1780x1294)

A warning message will pop up with the message "This workbook cannot be saved in the selected file format because it contains multiple sheets.".

Bypass the warning message by clicking on `OK`. Microsoft Excel tells you that it can't export all sheets to a single CSV. Only your active sheet will be exported to a CSV file. This is what we want.

![Confirm with OK](/learn_images/excel/export-selection-csv/confirm-to-export-active-sheet.png#1512x1076)


## Edit it with a CSV Editor

If you need to further clean your CSV data, use a [CSV Editor](/csv-editor) such as [Datablist](/csv-editor).

Datablist provides advanced tools to [deduplicate data](/features/duplicates-remover), [consolidate several CSV files](/features/combine-csv-files), or [clean your data](/how-to/clean-web-scraping-data-online).







Exporting a sheet from Microsoft Excel to a CSV file is straightforward. But how to export selected cells to a CSV file?
The short answer is you can&amp#39;t export selected cells in CSV quickly. Exporting a selection to a CSV file or another Excel file is not directly possible. Here is an alternative solution: copy the data to another sheet and export the sheet to a CSV file. 
Step 1: Copy-Paste selected cells
Select the cells you want to export. And copy the data to your clipboard (ctrl+c or Menu Edit =&ampgt; Copy)
File -&ampgt; Export To -&ampgt; CSV
Export only visible cells (without hidden columns)
If you have hidden columns, Excel will add their content to your clipboard. This is not what we want. We want to export only our selected and visible cells.
To prevent hidden columns to be copied, open the Go to Special menu in Find &ampamp; Select.
Find &ampamp; Select -&ampgt; Go to Special
And select Visible cells only. It will tell Microsoft Excel we only want to copy the visible cells.
Go to special -&ampgt; Visible cells only
Step 2: Create a new sheet and paste your data
Then, create a new sheet and paste your data (ctrl + v or Menu Edit =&ampgt; Paste)
Paste data in a new sheet
Add columns to have CSV headers
CSV files are used to store structured data, and CSV headers add context to know what your CSV contains.
CSV headers are optional. But if you need to import your CSV into another software or clean it with a CSV editor, you better have them.
Just copy - paste the columns from the first sheet or write them directly as the first line in the new sheet.
Paste columns in the new sheet
Step 3: Export sheet to CSV
The last step is to export the new Excel sheet to a CSV file. Once you are on the new sheet, select Save As in the File menu.
Important: Set the active sheet to the sheet to export!

File -&ampgt; Save As
Then change the File format to CSV UTF-8 (Comma-delimited) (.csv). And click Save.
Save As with CSV format
A warning message will pop up with the message &ampquot;This workbook cannot be saved in the selected file format because it contains multiple sheets.&ampquot;.
Bypass the warning message by clicking on OK. Microsoft Excel tells you that it can&amp#39;t export all sheets to a single CSV. Only your active sheet will be exported to a CSV file. This is what we want.
Confirm with OK
Edit it with a CSV Editor
If you need to further clean your CSV data, use a CSV Editor such as Datablist.
Datablist provides advanced tools to deduplicate data, consolidate several CSV files, or clean your data.


Export Microsoft Excel selection into a CSV file

To convert a <code>.xlsx</code> or <code>.xls</code> file into a <code>.csv</code> file, Microsoft Excel is the perfect tool.
First, load your <code>.xlsx</code> or <code>.xls</code> file with Excel. Then, use the File -> Save As menu action.
<img src="/learn_images/csv/export_excel_to_csv/microsoft_excel_convert_to_csv_save_as.png#1488x966" alt="File -> Save As">
Choose where you want to save your CSV file and select CSV UTF-8 (Comma delimited) (.csv) in the File format dropdown.
<img src="/learn_images/csv/export_excel_to_csv/microsoft_excel_convert_to_csv_select_format.png#1378x892" alt="Select CSV in file formats">
Finally, click the Save button.


To convert a `.xlsx` or `.xls` file into a `.csv` file, Microsoft Excel is the perfect tool.

First, load your `.xlsx` or `.xls` file with Excel. Then, use the **File -> Save As** menu action.

![File -> Save As](/learn_images/csv/export_excel_to_csv/microsoft_excel_convert_to_csv_save_as.png#1488x966)

Choose where you want to save your CSV file and select **CSV UTF-8 (Comma delimited) (.csv)** in the File format dropdown.

![Select CSV in file formats](/learn_images/csv/export_excel_to_csv/microsoft_excel_convert_to_csv_select_format.png#1378x892)


Finally, click the **Save** button.


To convert a .xlsx or .xls file into a .csv file, Microsoft Excel is the perfect tool.
First, load your .xlsx or .xls file with Excel. Then, use the File -&ampgt; Save As menu action.
File -&ampgt; Save As
Choose where you want to save your CSV file and select CSV UTF-8 (Comma delimited) (.csv) in the File format dropdown.
Select CSV in file formats
Finally, click the Save button.


Export Microsoft Excel spreadsheet into a CSV file

When it comes to storing data online, spreadsheets and databases are two possible options. Understanding the differences between them is crucial to find the best tool for your task.
Spreadsheets (Excel and Google Sheets) are easy to use and inexpensive. They are the go-to solution when it comes to simple analytics and data manipulation. For example, spreadsheet tools offer analytics features to display charts or create pivot tables.
Databases are designed to handle larger datasets and structured data. They offer built-in data constraints to force field format or to avoid duplicate information. Databases have powerful query languages to search and filter their data. These features are powerful but add complexity compared to spreadsheets. As a result, databases have a steep learning curve.
If you need to connect to your data programmatically, databases are the go-to solution. Thanks to its data structure, a database ensure the data is valid before saving it.
If you need to store less than 1 million records and don't need to connect your data to an application, spreadsheets do the job.


When it comes to storing data online, spreadsheets and databases are two possible options. Understanding the differences between them is crucial to find the best tool for your task.

Spreadsheets (Excel and Google Sheets) are easy to use and inexpensive. They are the go-to solution when it comes to simple analytics and data manipulation. For example, spreadsheet tools offer analytics features to display charts or create pivot tables.

Databases are designed to handle larger datasets and structured data. They offer built-in data constraints to force field format or to avoid duplicate information. Databases have powerful query languages to search and filter their data. These features are powerful but add complexity compared to spreadsheets. As a result, databases have a steep learning curve.

If you need to connect to your data programmatically, databases are the go-to solution. Thanks to its data structure, a database ensure the data is valid before saving it.

If you need to store less than 1 million records and don't need to connect your data to an application, spreadsheets do the job.


When it comes to storing data online, spreadsheets and databases are two possible options. Understanding the differences between them is crucial to find the best tool for your task.
Spreadsheets (Excel and Google Sheets) are easy to use and inexpensive. They are the go-to solution when it comes to simple analytics and data manipulation. For example, spreadsheet tools offer analytics features to display charts or create pivot tables.
Databases are designed to handle larger datasets and structured data. They offer built-in data constraints to force field format or to avoid duplicate information. Databases have powerful query languages to search and filter their data. These features are powerful but add complexity compared to spreadsheets. As a result, databases have a steep learning curve.
If you need to connect to your data programmatically, databases are the go-to solution. Thanks to its data structure, a database ensure the data is valid before saving it.
If you need to store less than 1 million records and don&amp#39;t need to connect your data to an application, spreadsheets do the job.


When to use a spreadsheet over a database?

Lead Deduplication, or <a href="/features/duplicates-remover">lead deduping</a>, is the process of finding, and merging duplicate records to have a clean list of unique entries.
A lead represents a person or a company. A duplicate lead is when there are multiple records for the same person or company.
Duplicate leads appear when you have several lead sources that pour into a single lead list. Examples of lead sources are lead magnets, webinars, newsletters, or manual entries.
👉 Check our <a href="/how-to/merge-duplicate-leads-without-losing-data">guide to remove duplicate leads from your list.</a>
The main steps for lead deduplication typically include:
<ol>
<li>
Data Collection: Gather the leads from various sources, such as online forms, CRM systems, marketing automation platforms, and manual entry. The best format to extract data from all sources is the <a href="/learn/csv/csv-meaning">CSV format</a>. Use our <a href="/csv-editor">CSV Editor</a> to import and consolidate all your files.
</li>
<li>
Data Cleansing: Cleanse and standardize the leads by removing any irrelevant or incomplete entries, correcting formatting issues, and normalizing data fields to ensure consistency.
</li>
<li>
Define Duplicate Scoring Algorithm: This involves setting what constitutes a duplicate. For lead deduplication, you want to define your matching criteria on unique identifiers: email address, phone number, or contact name. For company leads the best identifier is the company website. Then, you can select how to compare values between leads. Do you want an exact match on names, or use fuzzy matching to match duplicate leads with slight changes.
</li>
<li>
Duplicate Identification: Run the deduplication algorithm on your leads. Duplicate groups are created with leads having a high similarity score.
</li>
<li>
Data Consolidation: Merge or update the duplicate records to create a single, consolidated, and accurate lead entry. This may involve merging contact details, activity history, and any associated information. <a href="/features/combine-csv-files">Data Consolidation</a> can be done automatically using rules or manually for conflicting leads.
</li>
</ol>
👉 Check our <a href="/how-to/merge-duplicate-leads-without-losing-data">free tool for lead deduplication.</a>
<img src="/learn_images/marketing/leads-deduplication/leads-deduplication.png#1078x1056" alt="Datablist Lead Deduplication Tool">
👉 Check our <a href="/how-to/merge-duplicate-leads-without-losing-data">free tool for lead deduplication.</a>


**Lead Deduplication, or [lead deduping](/features/duplicates-remover), is the process of finding, and merging duplicate records to have a clean list of unique entries.**

A lead represents a person or a company. A duplicate lead is when there are multiple records for the same person or company.

Duplicate leads appear when you have several lead sources that pour into a single lead list. Examples of lead sources are lead magnets, webinars, newsletters, or manual entries.

👉 Check our [guide to remove duplicate leads from your list.](/how-to/merge-duplicate-leads-without-losing-data)

The main steps for lead deduplication typically include:

1. Data Collection: Gather the leads from various sources, such as online forms, CRM systems, marketing automation platforms, and manual entry. The best format to extract data from all sources is the [CSV format](/learn/csv/csv-meaning). Use our [CSV Editor](/csv-editor) to import and consolidate all your files.

1. Data Cleansing: Cleanse and standardize the leads by removing any irrelevant or incomplete entries, correcting formatting issues, and normalizing data fields to ensure consistency.

1. Define Duplicate Scoring Algorithm: This involves setting what constitutes a duplicate. For lead deduplication, you want to define your matching criteria on unique identifiers: email address, phone number, or contact name. For company leads the best identifier is the company website. Then, you can select how to compare values between leads. Do you want an exact match on names, or use fuzzy matching to match duplicate leads with slight changes. 

1. Duplicate Identification: Run the deduplication algorithm on your leads. Duplicate groups are created with leads having a high similarity score.

1. Data Consolidation: Merge or update the duplicate records to create a single, consolidated, and accurate lead entry. This may involve merging contact details, activity history, and any associated information. [Data Consolidation](/features/combine-csv-files) can be done automatically using rules or manually for conflicting leads.

👉 Check our [free tool for lead deduplication.](/how-to/merge-duplicate-leads-without-losing-data)

![Datablist Lead Deduplication Tool](/learn_images/marketing/leads-deduplication/leads-deduplication.png#1078x1056)

👉 Check our [free tool for lead deduplication.](/how-to/merge-duplicate-leads-without-losing-data)

Lead Deduplication, or lead deduping, is the process of finding, and merging duplicate records to have a clean list of unique entries.
A lead represents a person or a company. A duplicate lead is when there are multiple records for the same person or company.
Duplicate leads appear when you have several lead sources that pour into a single lead list. Examples of lead sources are lead magnets, webinars, newsletters, or manual entries.
👉 Check our guide to remove duplicate leads from your list.
The main steps for lead deduplication typically include:
Data Collection: Gather the leads from various sources, such as online forms, CRM systems, marketing automation platforms, and manual entry. The best format to extract data from all sources is the CSV format. Use our CSV Editor to import and consolidate all your files.

Data Cleansing: Cleanse and standardize the leads by removing any irrelevant or incomplete entries, correcting formatting issues, and normalizing data fields to ensure consistency.

Define Duplicate Scoring Algorithm: This involves setting what constitutes a duplicate. For lead deduplication, you want to define your matching criteria on unique identifiers: email address, phone number, or contact name. For company leads the best identifier is the company website. Then, you can select how to compare values between leads. Do you want an exact match on names, or use fuzzy matching to match duplicate leads with slight changes. 

Duplicate Identification: Run the deduplication algorithm on your leads. Duplicate groups are created with leads having a high similarity score.

Data Consolidation: Merge or update the duplicate records to create a single, consolidated, and accurate lead entry. This may involve merging contact details, activity history, and any associated information. Data Consolidation can be done automatically using rules or manually for conflicting leads.


👉 Check our free tool for lead deduplication.
Datablist Lead Deduplication Tool
👉 Check our free tool for lead deduplication.


When to dedupe leads?

Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!
The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist <a href="/csv-editor">CSV editor</a>.
<blockquote>
Notes: The code works also with Firefox, Safari, etc.
</blockquote>
<h2 id="user-content-step-1-run-javascript-code-in-google-chrome-developer-tools"><a href="#step-1-run-javascript-code-in-google-chrome-developer-tools" aria-hidden="true" tabindex="-1"></a>Step 1: Run JavaScript code in Google Chrome Developer Tools</h2>
<ul>
<li>Open Google Chrome Developer Tools with <code>Cmd + Opt + i</code> (Mac) or <code>F12</code> (Windows). And click on the <code>Console</code> tab.</li>
<li>Copy-Paste the following JavaScript code and press <code>Enter</code>.</li>
</ul>
<pre><code>const results = [
 ['Url', 'Anchor Text', 'External']
];
var urls = document.getElementsByTagName('a');
for (urlIndex in urls) {
 const url = urls[urlIndex]
 const externalLink = url.host !== window.location.host
 if(url.href &#x26;&#x26; url.href.indexOf('://')!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=>{
 return line.map((cell)=>{
 if(typeof(cell)==='boolean') return cell ? 'TRUE': 'FALSE'
 if(!cell) return ''
 let value = cell.replace(/[\f\n\v]*\n\s*/g, "\n").replace(/[\t\f ]+/g, ' ');
 value = value.replace(/\t/g, ' ').trim();
 return `"${value}"`
 }).join('\t')
}).join("\n");
console.log(csvContent)
</code></pre>
<img src="/learn_images/scraping/extract-urls-webpage/copy-js-code.png#1938x694" alt="Open Google Developer Tools">
<h2 id="user-content-step-2-copy-paste-exported-urls-into-a-csv-file-or-spreadsheet-tools"><a href="#step-2-copy-paste-exported-urls-into-a-csv-file-or-spreadsheet-tools" aria-hidden="true" tabindex="-1"></a>Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools</h2>
The JavaScript code will extract all URLs from the webpage with the following values:
<ul>
<li>URL - The link URL</li>
<li>Anchor Text - The label associated with the link. Called "Anchor Text".</li>
<li>External - A boolean (<code>TRUE</code>, <code>FALSE</code>)</li>
</ul>
The result is printed on the console using <a href="/learn/csv/csv-meaning">CSV format</a>. You can copy-paste the result into spreadsheet tools or with our <a href="/csv-editor">online CSV editor</a>. Or create a new text file with your text editor and save the file with a <code>.csv</code> extension.
Google automatically adds a <code>Copy</code> button when you get a lot of results.
<img src="/learn_images/scraping/extract-urls-webpage/copy-results.png#1790x664" alt="Open Google Results">
<h2 id="user-content-step-3-filter-csv-data-to-get-relevant-links"><a href="#step-3-filter-csv-data-to-get-relevant-links" aria-hidden="true" tabindex="-1"></a>Step 3: Filter CSV data to get relevant links</h2>
To filter quickly your links, open Datablist <a href="/csv-editor">CSV editor</a>. Create a collection and paste your CSV data. Check the <a href="#example-with-google-results">Google</a> and <a href="#example-with-amazon-results">Amazon</a> examples below for a more step-by-step guide.
<img src="/learn_images/scraping/extract-urls-webpage/create-collection.png#1064x668" alt="Create new collection">
Select 'First row contains header' and click on the "+" button for each column to import them.
<img src="/learn_images/scraping/extract-urls-webpage/paste-results.png#2537x1346" alt="Paste results">
Then you will be able to use text search or filter on a specific property.
You can use Datablist to <a href="/features/combine-csv-files">consolidate all your CSV data</a>. And Datablist comes with an algorithm for <a href="/features/duplicates-remover">deduplication</a> to have a clean list or URLs.
<h2 id="user-content-example-with-google-results"><a href="#example-with-google-results" aria-hidden="true" tabindex="-1"></a>Example with Google Results</h2>
This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.
<h3 id="user-content-step-1-search-for-a-google-term-that-you-want-to-extract-links"><a href="#step-1-search-for-a-google-term-that-you-want-to-extract-links" aria-hidden="true" tabindex="-1"></a>Step 1: Search for a Google term that you want to extract links.</h3>
A nice tip is to add the <code>&#x26;num=100</code> parameter to the URL to force Google into showing 100 results per page.
<img src="/learn_images/scraping/extract-urls-webpage/google/google-num-param.png#1696x458" alt="Force 100 results">
<img src="/learn_images/scraping/extract-urls-webpage/google/step-1-google-results.png#1934x1300" alt="Open Google Results">
<h3 id="user-content-step-2-copy-paste-the-code-and-run-it"><a href="#step-2-copy-paste-the-code-and-run-it" aria-hidden="true" tabindex="-1"></a>Step 2: Copy-paste the code and run it</h3>
Open the <code>console</code> in Google Developer Tools, paste the JavaScript code and press <code>Enter</code>.
<img src="/learn_images/scraping/extract-urls-webpage/google/step2-copy-paste-code.png#2532x1235" alt="Copy-paste code">
Copy the results. Select the line and <code>ctrl + c</code> or use the <code>Copy</code> button when Chrome shows it.
<img src="/learn_images/scraping/extract-urls-webpage/google/step3-run-code.png#2532x1344" alt="Run JavaScript code">
<h3 id="user-content-step-3-filter-results"><a href="#step-3-filter-results" aria-hidden="true" tabindex="-1"></a>Step 3: Filter results</h3>
Open Datablist <a href="/csv-editor">CSV editor</a>, create a collection, and paste the copied URLs.
<blockquote>
Note: Don't forget to select the option <code>First row contains headers</code> and to click the "+" button as shown.
</blockquote>
<img src="/learn_images/scraping/extract-urls-webpage/google/step4-create-collection-datablist.png#2537x1346" alt="Copy-Paste results in Datablist">
Once imported. Click "Filter on property" in the "External" column.
<img src="/learn_images/scraping/extract-urls-webpage/google/step5-filter-1.png#2760x1750" alt="Create Filters">
And filter items with <code>External</code> to <code>TRUE</code> and <code>Url</code> that does not contain <code>google</code>. Check our guide on <a href="/how-to/clean-web-scraping-data-online">how to clean your scraped data</a>
<img src="/learn_images/scraping/extract-urls-webpage/google/step5-filter-2.png#2533x1355" alt="Filter URLs outside Google">
And here are your external links:
<img src="/learn_images/scraping/extract-urls-webpage/google/step6-results.png#2371x1432" alt="Google Links results">
<h2 id="user-content-example-with-amazon-results"><a href="#example-with-amazon-results" aria-hidden="true" tabindex="-1"></a>Example with Amazon Results</h2>
This is a step-by-step example with the Amazon products page. We want to extract all links from this page.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-speakers-results.png#1926x1588" alt="Amazon products listing">
Open the Chrome Developer Tools:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-open-developer-tools.png#1930x1650" alt="Open Chrome Developer Tools">
Copy-paste the JavaScript code:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-copy-paste-code.png#2162x1830" alt="Copy-Paste code">
And run it by pressing <code>Enter</code>:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-run-code.png#2162x1830" alt="Run code">
With Datablist <a href="/csv-editor">CSV editor</a>, create a collection.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-create-collection.png#2185x1098" alt="Create collection">
Paste your scrapped URLs:
<img src="/learn_images/scraping/extract-urls-webpage/amazon/import-from-pasting.png#2195x1202" alt="Paste scrapped URLs">
Your scrapped URLs are listed on the tool. Filter the links, <a href="/features/duplicates-remover">deduplicate the results</a>, and <a href="/how-to/clean-web-scraping-data-online">clean your scraped data</a>.
<img src="/learn_images/scraping/extract-urls-webpage/amazon/amazon-results.png#2643x1366" alt="Scrapped Amazon URLs">


Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!

**The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist [CSV editor](/csv-editor).**

> **Notes**: The code works also with Firefox, Safari, etc.

## Step 1: Run JavaScript code in Google Chrome Developer Tools

- Open Google Chrome Developer Tools with `Cmd + Opt + i` (Mac) or `F12` (Windows). And click on the `Console` tab.
- Copy-Paste the following JavaScript code and press `Enter`.

```
const results = [
    ['Url', 'Anchor Text', 'External']
];
var urls = document.getElementsByTagName('a');
for (urlIndex in urls) {
    const url = urls[urlIndex]
    const externalLink = url.host !== window.location.host
    if(url.href && url.href.indexOf('://')!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=>{
    return line.map((cell)=>{
        if(typeof(cell)==='boolean') return cell ? 'TRUE': 'FALSE'
        if(!cell) return ''
        let value = cell.replace(/[\f\n\v]*\n\s*/g, "\n").replace(/[\t\f ]+/g, ' ');
        value = value.replace(/\t/g, ' ').trim();
        return `"${value}"`
    }).join('\t')
}).join("\n");
console.log(csvContent)
```

![Open Google Developer Tools](/learn_images/scraping/extract-urls-webpage/copy-js-code.png#1938x694)


## Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools

The JavaScript code will extract all URLs from the webpage with the following values:

- **URL** - The link URL
- **Anchor Text** - The label associated with the link. Called "Anchor Text".
- **External** - A boolean (`TRUE`, `FALSE`)

The result is printed on the console using [CSV format](/learn/csv/csv-meaning). You can copy-paste the result into spreadsheet tools or with our [online CSV editor](/csv-editor). Or create a new text file with your text editor and save the file with a `.csv` extension.

Google automatically adds a `Copy` button when you get a lot of results.

![Open Google Results](/learn_images/scraping/extract-urls-webpage/copy-results.png#1790x664)


## Step 3: Filter CSV data to get relevant links

To filter quickly your links, open Datablist [CSV editor](/csv-editor). Create a collection and paste your CSV data. Check the [Google](#example-with-google-results) and [Amazon](#example-with-amazon-results) examples below for a more step-by-step guide.

![Create new collection](/learn_images/scraping/extract-urls-webpage/create-collection.png#1064x668)

Select 'First row contains header' and click on the "+" button for each column to import them.

![Paste results](/learn_images/scraping/extract-urls-webpage/paste-results.png#2537x1346)

Then you will be able to use text search or filter on a specific property.
You can use Datablist to [consolidate all your CSV data](/features/combine-csv-files). And Datablist comes with an algorithm for [deduplication](/features/duplicates-remover) to have a clean list or URLs.


## Example with Google Results

This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.

### Step 1: Search for a Google term that you want to extract links.

**A nice tip is to add the `&num=100` parameter to the URL to force Google into showing 100 results per page.**

![Force 100 results](/learn_images/scraping/extract-urls-webpage/google/google-num-param.png#1696x458)

![Open Google Results](/learn_images/scraping/extract-urls-webpage/google/step-1-google-results.png#1934x1300)


### Step 2: Copy-paste the code and run it

Open the `console` in Google Developer Tools, paste the JavaScript code and press `Enter`.

![Copy-paste code](/learn_images/scraping/extract-urls-webpage/google/step2-copy-paste-code.png#2532x1235)

Copy the results. Select the line and `ctrl + c` or use the `Copy` button when Chrome shows it.

![Run JavaScript code](/learn_images/scraping/extract-urls-webpage/google/step3-run-code.png#2532x1344)

### Step 3: Filter results

Open Datablist [CSV editor](/csv-editor), create a collection, and paste the copied URLs.

> **Note**: Don't forget to select the option `First row contains headers` and to click the "+" button as shown.

![Copy-Paste results in Datablist](/learn_images/scraping/extract-urls-webpage/google/step4-create-collection-datablist.png#2537x1346)

Once imported. Click "Filter on property" in the "External" column.

![Create Filters](/learn_images/scraping/extract-urls-webpage/google/step5-filter-1.png#2760x1750)


And filter items with `External` to `TRUE` and `Url` that does not contain `google`. Check our guide on [how to clean your scraped data](/how-to/clean-web-scraping-data-online)

![Filter URLs outside Google](/learn_images/scraping/extract-urls-webpage/google/step5-filter-2.png#2533x1355)

And here are your external links:

![Google Links results](/learn_images/scraping/extract-urls-webpage/google/step6-results.png#2371x1432)

## Example with Amazon Results

This is a step-by-step example with the Amazon products page. We want to extract all links from this page.

![Amazon products listing](/learn_images/scraping/extract-urls-webpage/amazon/amazon-speakers-results.png#1926x1588)

Open the Chrome Developer Tools:

![Open Chrome Developer Tools](/learn_images/scraping/extract-urls-webpage/amazon/amazon-open-developer-tools.png#1930x1650)

Copy-paste the JavaScript code:

![Copy-Paste code](/learn_images/scraping/extract-urls-webpage/amazon/amazon-copy-paste-code.png#2162x1830)

And run it by pressing `Enter`:

![Run code](/learn_images/scraping/extract-urls-webpage/amazon/amazon-run-code.png#2162x1830)

With Datablist [CSV editor](/csv-editor), create a collection.

![Create collection](/learn_images/scraping/extract-urls-webpage/amazon/amazon-create-collection.png#2185x1098)

Paste your scrapped URLs:

![Paste scrapped URLs](/learn_images/scraping/extract-urls-webpage/amazon/import-from-pasting.png#2195x1202)

Your scrapped URLs are listed on the tool. Filter the links, [deduplicate the results](/features/duplicates-remover), and [clean your scraped data](/how-to/clean-web-scraping-data-online).

![Scrapped Amazon URLs](/learn_images/scraping/extract-urls-webpage/amazon/amazon-results.png#2643x1366)








Here is a quick JavaScript snippet to extract all URLs from a webpage fast with Google Chrome Developer Tools. No Browser Extension is required!
The JavaScript code generates a list of URLs in CSV format with the anchor texts, and a boolean to know if the URL is internal or external to the current website. Just copy-paste the results into Google Sheets or with the Datablist CSV editor.
Notes: The code works also with Firefox, Safari, etc.

Step 1: Run JavaScript code in Google Chrome Developer Tools
Open Google Chrome Developer Tools with Cmd + Opt + i (Mac) or F12 (Windows). And click on the Console tab.
Copy-Paste the following JavaScript code and press Enter.

const results = [
    [&#39Url&#39, &#39Anchor Text&#39, &#39External&#39]
];
var urls = document.getElementsByTagName(&#39a&#39);
for (urlIndex in urls) {
    const url = urls[urlIndex]
    const externalLink = url.host !== window.location.host
    if(url.href &amp&amp url.href.indexOf(&#39://&#39)!==-1) results.push([url.href, url.text, externalLink]) // url.rel
}
const csvContent = results.map((line)=&gt{
    return line.map((cell)=&gt{
        if(typeof(cell)===&#39boolean&#39) return cell ? &#39TRUE&#39: &#39FALSE&#39
        if(!cell) return &#39&#39
        let value = cell.replace(/[\f\n\v]*\n\s*/g, &quot\n&quot).replace(/[\t\f ]+/g, &#39 &#39);
        value = value.replace(/\t/g, &#39 &#39).trim();
        return `&quot${value}&quot`
    }).join(&#39\t&#39)
}).join(&quot\n&quot);
console.log(csvContent)
Open Google Developer Tools
Step 2: Copy-paste exported URLs into a CSV file or spreadsheet tools
The JavaScript code will extract all URLs from the webpage with the following values:
URL - The link URL
Anchor Text - The label associated with the link. Called &ampquot;Anchor Text&ampquot;.
External - A boolean (TRUE, FALSE)

The result is printed on the console using CSV format. You can copy-paste the result into spreadsheet tools or with our online CSV editor. Or create a new text file with your text editor and save the file with a .csv extension.
Google automatically adds a Copy button when you get a lot of results.
Open Google Results
Step 3: Filter CSV data to get relevant links
To filter quickly your links, open Datablist CSV editor. Create a collection and paste your CSV data. Check the Google and Amazon examples below for a more step-by-step guide.
Create new collection
Select &amp#39;First row contains header&amp#39; and click on the &ampquot;+&ampquot; button for each column to import them.
Paste results
Then you will be able to use text search or filter on a specific property.
You can use Datablist to consolidate all your CSV data. And Datablist comes with an algorithm for deduplication to have a clean list or URLs.
Example with Google Results
This is a step-by-step example with the Google results page. We want to extract all external links from a Google search result.
Step 1: Search for a Google term that you want to extract links.
A nice tip is to add the &ampamp;num=100 parameter to the URL to force Google into showing 100 results per page.
Force 100 results
Open Google Results
Step 2: Copy-paste the code and run it
Open the console in Google Developer Tools, paste the JavaScript code and press Enter.
Copy-paste code
Copy the results. Select the line and ctrl + c or use the Copy button when Chrome shows it.
Run JavaScript code
Step 3: Filter results
Open Datablist CSV editor, create a collection, and paste the copied URLs.
Note: Don&amp#39;t forget to select the option First row contains headers and to click the &ampquot;+&ampquot; button as shown.

Copy-Paste results in Datablist
Once imported. Click &ampquot;Filter on property&ampquot; in the &ampquot;External&ampquot; column.
Create Filters
And filter items with External to TRUE and Url that does not contain google. Check our guide on how to clean your scraped data
Filter URLs outside Google
And here are your external links:
Google Links results
Example with Amazon Results
This is a step-by-step example with the Amazon products page. We want to extract all links from this page.
Amazon products listing
Open the Chrome Developer Tools:
Open Chrome Developer Tools
Copy-paste the JavaScript code:
Copy-Paste code
And run it by pressing Enter:
Run code
With Datablist CSV editor, create a collection.
Create collection
Paste your scrapped URLs:
Paste scrapped URLs
Your scrapped URLs are listed on the tool. Filter the links, deduplicate the results, and clean your scraped data.
Scrapped Amazon URLs


Extracy all URLs from a webpage

Records deduplication, also known as data deduplication or <a href="/how-to/data-matching-tool">record matching</a>, is the process of identifying and eliminating duplicate records or entries within a database or dataset. It's a critical <a href="/how-to/data-cleaning">data cleansing</a> technique used to ensure data accuracy and consistency.
<h2 id="user-content-how-does-record-deduplication-work"><a href="#how-does-record-deduplication-work" aria-hidden="true" tabindex="-1"></a>How does Record Deduplication work?</h2>
Record deduplication typically involves two steps:
<ul>
<li>Identifying duplicate records or <a href="/how-to/data-matching-tool">Data Matching</a>: The first step is to identify duplicate records in the dataset. This can be done using a variety of methods, such as comparing unique identifiers (e.g., customer IDs, product IDs), or using phonetic/fuzzy algorithms to identify records that are similar to each other.</li>
<li>Removing duplicate records: Once duplicate records have been identified, they can be removed from the dataset. This can be done manually, or using automated tools. When record data is similar or complementary, the merging process is a nondestructive step. When duplicate records have conflicting data, the merging process must offer text concatenation or a rule to drop conflicting data to be able to merge all fields into a single record.</li>
</ul>
<h2 id="user-content-how-to-identify-duplicate-records"><a href="#how-to-identify-duplicate-records" aria-hidden="true" tabindex="-1"></a>How to identify duplicate records?</h2>
There are two main types of data matching:
<ul>
<li>Exact record matching: Exact deduplication identifies and removes records that are exactly the same. This is the simplest type of deduplication.</li>
<li>Field-specific matching: This is a more complex type of deduplication. It requires the use of algorithms to identify similar records by focusing on specific fields like names, addresses, or phone numbers.</li>
</ul>
For field-specific matching, several algorithms exist:
<ul>
<li>Exact matching: Identifies records that have identical values for the specific fields.</li>
<li>Phonetic matching: Algorithm to match values by their pronunciation. Most phonetic algorithms work best with English words. The <a href="https://en.wikipedia.org/wiki/Soundex" target="_blank" rel="nofollow">Soundex</a> and <a href="/learn/data-cleaning/double-metaphone">Double Metaphone</a> are among the best-known phonetic algorithms.</li>
<li>Fuzzy Matching: Uses algorithms to find records that are similar but not necessarily identical, accommodating variations in spelling, formatting, or other data discrepancies. Two famous fuzzy matching algorithms are the <a href="/learn/data-cleaning/fuzzy-matching-levenshtein-distance">Levenshtein Distance</a> and <a href="/learn/data-cleaning/fuzzy-matching-jaro-winkler-distance">Jaro-Winkler Distance</a>.</li>
</ul>
<h2 id="user-content-why-is-record-deduplication-important"><a href="#why-is-record-deduplication-important" aria-hidden="true" tabindex="-1"></a>Why is Record Deduplication Important?</h2>
Data or record deduplication is crucial for several reasons:
<ul>
<li>Data Quality: Removing duplicates improves data accuracy, integrity, and consistency, preventing errors and confusion.</li>
<li>Efficient Operations: It reduces the storage space required, speeds up data retrieval, and optimizes database performance.</li>
<li>Compliance: In industries with regulatory requirements, deduplication helps maintain data integrity and meet compliance standards.</li>
</ul>
<h2 id="user-content-what-tool-is-best-to-perform-records-deduplication"><a href="#what-tool-is-best-to-perform-records-deduplication" aria-hidden="true" tabindex="-1"></a>What tool is best to perform records deduplication?</h2>
Datablist is a <a href="/csv-editor">free csv editor</a> with powerful <a href="/how-to/data-cleaning">data cleansing</a> features. Datablist <a href="/features/duplicates-remover">Duplicates Finder</a> implements Exact, Phonetic, and Fuzzy Matching algorithms to detect duplicate records across your datasets.
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>
<img src="/learn_images/data-cleaning/deduplication/data-matching.png#1922x1302" alt="Data Matching">
<img src="/learn_images/data-cleaning/deduplication/data-deduplication.png#2160x1370" alt="Data Deduplication">
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>
<a href="/how-to/remove-csv-duplicates">👉 Read our guide on CSV deduplication.</a>


**Records deduplication**, also known as **data deduplication** or [record matching](/how-to/data-matching-tool), is the process of identifying and eliminating duplicate records or entries within a database or dataset. It's a critical [data cleansing](/how-to/data-cleaning) technique used to ensure data accuracy and consistency.

## How does Record Deduplication work?

Record deduplication typically involves two steps:

- Identifying duplicate records or [**Data Matching**](/how-to/data-matching-tool): The first step is to identify duplicate records in the dataset. This can be done using a variety of methods, such as comparing unique identifiers (e.g., customer IDs, product IDs), or using phonetic/fuzzy algorithms to identify records that are similar to each other.
- **Removing duplicate records**: Once duplicate records have been identified, they can be removed from the dataset. This can be done manually, or using automated tools. When record data is similar or complementary, the merging process is a nondestructive step. When duplicate records have conflicting data, the merging process must offer text concatenation or a rule to drop conflicting data to be able to merge all fields into a single record.

## How to identify duplicate records?

There are two main types of data matching:

- **Exact record matching**: Exact deduplication identifies and removes records that are exactly the same. This is the simplest type of deduplication.
- **Field-specific matching**: This is a more complex type of deduplication. It requires the use of algorithms to identify similar records by focusing on specific fields like names, addresses, or phone numbers. 

For field-specific matching, several algorithms exist:

- **Exact matching**: Identifies records that have identical values for the specific fields.
- **Phonetic matching**: Algorithm to match values by their pronunciation. Most phonetic algorithms work best with English words. The [Soundex](https://en.wikipedia.org/wiki/Soundex) and [Double Metaphone](/learn/data-cleaning/double-metaphone) are among the best-known phonetic algorithms.
- **Fuzzy Matching**: Uses algorithms to find records that are similar but not necessarily identical, accommodating variations in spelling, formatting, or other data discrepancies. Two famous fuzzy matching algorithms are the [Levenshtein Distance](/learn/data-cleaning/fuzzy-matching-levenshtein-distance) and [Jaro-Winkler Distance](/learn/data-cleaning/fuzzy-matching-jaro-winkler-distance).



## Why is Record Deduplication Important?

Data or record deduplication is crucial for several reasons:

- **Data Quality**: Removing duplicates improves data accuracy, integrity, and consistency, preventing errors and confusion.
- **Efficient Operations**: It reduces the storage space required, speeds up data retrieval, and optimizes database performance.
- **Compliance**: In industries with regulatory requirements, deduplication helps maintain data integrity and meet compliance standards.

## What tool is best to perform records deduplication?

Datablist is a [free csv editor](/csv-editor) with powerful [data cleansing](/how-to/data-cleaning) features. Datablist [Duplicates Finder](/features/duplicates-remover) implements Exact, Phonetic, and Fuzzy Matching algorithms to detect duplicate records across your datasets.

[👉 Open Datablist (it's free)](https://app.datablist.com)

![Data Matching](/learn_images/data-cleaning/deduplication/data-matching.png#1922x1302)

![Data Deduplication](/learn_images/data-cleaning/deduplication/data-deduplication.png#2160x1370)

[👉 Open Datablist (it's free)](https://app.datablist.com)

[👉 Read our guide on CSV deduplication.](/how-to/remove-csv-duplicates)

Records deduplication, also known as data deduplication or record matching, is the process of identifying and eliminating duplicate records or entries within a database or dataset. It&amp#39;s a critical data cleansing technique used to ensure data accuracy and consistency.
How does Record Deduplication work?
Record deduplication typically involves two steps:
Identifying duplicate records or Data Matching: The first step is to identify duplicate records in the dataset. This can be done using a variety of methods, such as comparing unique identifiers (e.g., customer IDs, product IDs), or using phonetic/fuzzy algorithms to identify records that are similar to each other.
Removing duplicate records: Once duplicate records have been identified, they can be removed from the dataset. This can be done manually, or using automated tools. When record data is similar or complementary, the merging process is a nondestructive step. When duplicate records have conflicting data, the merging process must offer text concatenation or a rule to drop conflicting data to be able to merge all fields into a single record.

How to identify duplicate records?
There are two main types of data matching:
Exact record matching: Exact deduplication identifies and removes records that are exactly the same. This is the simplest type of deduplication.
Field-specific matching: This is a more complex type of deduplication. It requires the use of algorithms to identify similar records by focusing on specific fields like names, addresses, or phone numbers. 

For field-specific matching, several algorithms exist:
Exact matching: Identifies records that have identical values for the specific fields.
Phonetic matching: Algorithm to match values by their pronunciation. Most phonetic algorithms work best with English words. The Soundex and Double Metaphone are among the best-known phonetic algorithms.
Fuzzy Matching: Uses algorithms to find records that are similar but not necessarily identical, accommodating variations in spelling, formatting, or other data discrepancies. Two famous fuzzy matching algorithms are the Levenshtein Distance and Jaro-Winkler Distance.

Why is Record Deduplication Important?
Data or record deduplication is crucial for several reasons:
Data Quality: Removing duplicates improves data accuracy, integrity, and consistency, preventing errors and confusion.
Efficient Operations: It reduces the storage space required, speeds up data retrieval, and optimizes database performance.
Compliance: In industries with regulatory requirements, deduplication helps maintain data integrity and meet compliance standards.

What tool is best to perform records deduplication?
Datablist is a free csv editor with powerful data cleansing features. Datablist Duplicates Finder implements Exact, Phonetic, and Fuzzy Matching algorithms to detect duplicate records across your datasets.
👉 Open Datablist (it&amp#39;s free)
Data Matching
Data Deduplication
👉 Open Datablist (it&amp#39;s free)
👉 Read our guide on CSV deduplication.


What is deduplication?

The Double Metaphone algorithm is a phonetic algorithm that is used to convert words into a phonetic code. The code can then be used to compare words and names that sound similar, even if they are spelled differently. The Double Metaphone algorithm is an improvement over the original Metaphone algorithm and was created by Lawrence Philips in 2000.
<h2 id="user-content-how-does-the-double-metaphone-algorithm-work"><a href="#how-does-the-double-metaphone-algorithm-work" aria-hidden="true" tabindex="-1"></a>How does the Double Metaphone algorithm work?</h2>
The Double Metaphone algorithm works by first removing all non-alphabetic characters from the word. Next, all vowels are removed, unless the word begins with a vowel. The remaining consonants are then processed according to a set of rules. These rules take into account the pronunciation of English consonants, as well as the way that consonants are often combined in English words.
The Double Metaphone algorithm generates two phonetic codes for each word. The first code is called the primary code and is used for most comparisons. The second code is called the secondary code and is used for comparisons where the primary code is not unique.
For example, the word "Smith" would be converted to the primary code "SMT" and the secondary code "XMT" by the Double Metaphone algorithm. This is because the "th" sound can be pronounced as either "S" or "X" in English, so both codes are generated.
<h2 id="user-content-what-are-the-benefits-of-using-the-double-metaphone-algorithm"><a href="#what-are-the-benefits-of-using-the-double-metaphone-algorithm" aria-hidden="true" tabindex="-1"></a>What are the benefits of using the Double Metaphone algorithm?</h2>
The Double Metaphone algorithm has a number of benefits:
<ul>
<li>
Language Agnostic: The Double Metaphone Algorithm works well with a multitude of languages, making it suitable for applications requiring cross-lingual compatibility.
</li>
<li>
Accurate Phonetic Encoding: It generates phonetic encodings that reflect the way words are spoken, enhancing the accuracy of searches and matches in databases and applications. The Double Metaphone algorithm is more accurate than Soundex and Metaphone at matching words that sound similar. This is because the Double Metaphone algorithm takes into account more of the pronunciation of English consonants.
</li>
<li>
Variation Handling: By providing primary and secondary codes, the algorithm accommodates the various ways a word might be pronounced, increasing the chances of capturing relevant results.
</li>
<li>
Improved Spell Correction: The Double Metaphone Algorithm aids in spell correction by identifying words that sound similar but are spelled differently.
</li>
<li>
Speed: The Double Metaphone algorithm is relatively fast, making it suitable for use in applications where speed is important.
</li>
</ul>
<h2 id="user-content-what-are-the-limitations-of-the-double-metaphone-algorithm"><a href="#what-are-the-limitations-of-the-double-metaphone-algorithm" aria-hidden="true" tabindex="-1"></a>What are the limitations of the Double Metaphone algorithm?</h2>
While powerful, the Double Metaphone Algorithm might not be perfectly accurate in all cases, especially for very short words or words borrowed from foreign languages that retain their original pronunciation.
<h2 id="user-content-how-to-use-double-metaphone-algorithm-to-find-duplicate-records"><a href="#how-to-use-double-metaphone-algorithm-to-find-duplicate-records" aria-hidden="true" tabindex="-1"></a>How to use Double Metaphone algorithm to find duplicate records?</h2>
The Double Metaphone algorithm is suited for <a href="/features/duplicates-remover">data deduplication</a>. This algorithm can be used to generate a code for each word in names, addresses, or any text fields. Then, the codes are compared to find duplicate records.
<a href="/">Datablist</a> is a free <a href="/data-editor">data editor</a> with powerful <a href="/how-to/data-cleaning">data-cleaning features</a>. Datablist <a href="/features/duplicates-remover">Duplicates Finder</a> implements the Double Metaphone to detect duplicate records across your datasets.
<img src="/learn_images/data-cleaning/double-metaphone/double-metaphone-deduplication.png#1336x1310" alt="Double Metaphone Algorithm">
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>


The Double Metaphone algorithm is a phonetic algorithm that is used to convert words into a phonetic code. The code can then be used to compare words and names that sound similar, even if they are spelled differently. The Double Metaphone algorithm is an improvement over the original Metaphone algorithm and was created by Lawrence Philips in 2000.

## How does the Double Metaphone algorithm work?

The Double Metaphone algorithm works by first removing all non-alphabetic characters from the word. Next, all vowels are removed, unless the word begins with a vowel. The remaining consonants are then processed according to a set of rules. These rules take into account the pronunciation of English consonants, as well as the way that consonants are often combined in English words.

The Double Metaphone algorithm generates two phonetic codes for each word. The first code is called the primary code and is used for most comparisons. The second code is called the secondary code and is used for comparisons where the primary code is not unique.

For example, the word "Smith" would be converted to the primary code "SMT" and the secondary code "XMT" by the Double Metaphone algorithm. This is because the "th" sound can be pronounced as either "S" or "X" in English, so both codes are generated.




## What are the benefits of using the Double Metaphone algorithm?

The Double Metaphone algorithm has a number of benefits:

- **Language Agnostic**: The Double Metaphone Algorithm works well with a multitude of languages, making it suitable for applications requiring cross-lingual compatibility.

- **Accurate Phonetic Encoding**: It generates phonetic encodings that reflect the way words are spoken, enhancing the accuracy of searches and matches in databases and applications. The Double Metaphone algorithm is more accurate than Soundex and Metaphone at matching words that sound similar. This is because the Double Metaphone algorithm takes into account more of the pronunciation of English consonants.

- **Variation Handling**: By providing primary and secondary codes, the algorithm accommodates the various ways a word might be pronounced, increasing the chances of capturing relevant results.

- **Improved Spell Correction**: The Double Metaphone Algorithm aids in spell correction by identifying words that sound similar but are spelled differently.

- **Speed**: The Double Metaphone algorithm is relatively fast, making it suitable for use in applications where speed is important.


## What are the limitations of the Double Metaphone algorithm?

While powerful, the Double Metaphone Algorithm might not be perfectly accurate in all cases, especially for very short words or words borrowed from foreign languages that retain their original pronunciation.

## How to use Double Metaphone algorithm to find duplicate records?

The Double Metaphone algorithm is suited for [data deduplication](/features/duplicates-remover). This algorithm can be used to generate a code for each word in names, addresses, or any text fields. Then, the codes are compared to find duplicate records.

[Datablist](/) is a free [data editor](/data-editor) with powerful [data-cleaning features](/how-to/data-cleaning). Datablist [Duplicates Finder](/features/duplicates-remover) implements the Double Metaphone to detect duplicate records across your datasets.

![Double Metaphone Algorithm](/learn_images/data-cleaning/double-metaphone/double-metaphone-deduplication.png#1336x1310)

[👉 Open Datablist (it's free)](https://app.datablist.com)



The Double Metaphone algorithm is a phonetic algorithm that is used to convert words into a phonetic code. The code can then be used to compare words and names that sound similar, even if they are spelled differently. The Double Metaphone algorithm is an improvement over the original Metaphone algorithm and was created by Lawrence Philips in 2000.
How does the Double Metaphone algorithm work?
The Double Metaphone algorithm works by first removing all non-alphabetic characters from the word. Next, all vowels are removed, unless the word begins with a vowel. The remaining consonants are then processed according to a set of rules. These rules take into account the pronunciation of English consonants, as well as the way that consonants are often combined in English words.
The Double Metaphone algorithm generates two phonetic codes for each word. The first code is called the primary code and is used for most comparisons. The second code is called the secondary code and is used for comparisons where the primary code is not unique.
For example, the word &ampquot;Smith&ampquot; would be converted to the primary code &ampquot;SMT&ampquot; and the secondary code &ampquot;XMT&ampquot; by the Double Metaphone algorithm. This is because the &ampquot;th&ampquot; sound can be pronounced as either &ampquot;S&ampquot; or &ampquot;X&ampquot; in English, so both codes are generated.
What are the benefits of using the Double Metaphone algorithm?
The Double Metaphone algorithm has a number of benefits:
Language Agnostic: The Double Metaphone Algorithm works well with a multitude of languages, making it suitable for applications requiring cross-lingual compatibility.

Accurate Phonetic Encoding: It generates phonetic encodings that reflect the way words are spoken, enhancing the accuracy of searches and matches in databases and applications. The Double Metaphone algorithm is more accurate than Soundex and Metaphone at matching words that sound similar. This is because the Double Metaphone algorithm takes into account more of the pronunciation of English consonants.

Variation Handling: By providing primary and secondary codes, the algorithm accommodates the various ways a word might be pronounced, increasing the chances of capturing relevant results.

Improved Spell Correction: The Double Metaphone Algorithm aids in spell correction by identifying words that sound similar but are spelled differently.

Speed: The Double Metaphone algorithm is relatively fast, making it suitable for use in applications where speed is important.


What are the limitations of the Double Metaphone algorithm?
While powerful, the Double Metaphone Algorithm might not be perfectly accurate in all cases, especially for very short words or words borrowed from foreign languages that retain their original pronunciation.
How to use Double Metaphone algorithm to find duplicate records?
The Double Metaphone algorithm is suited for data deduplication. This algorithm can be used to generate a code for each word in names, addresses, or any text fields. Then, the codes are compared to find duplicate records.
Datablist is a free data editor with powerful data-cleaning features. Datablist Duplicates Finder implements the Double Metaphone to detect duplicate records across your datasets.
Double Metaphone Algorithm
👉 Open Datablist (it&amp#39;s free)


What is the Double Metaphone algorithm?

The Jaro-Winkler Distance is a similarity measure that quantifies the difference between two strings, often used in the context of record linkage, data deduplication, and string matching. It's an extension of the Jaro Distance metric and takes into account prefixes of matching characters.
<h2 id="user-content-how-does-the-jaro-winkler-distance-work"><a href="#how-does-the-jaro-winkler-distance-work" aria-hidden="true" tabindex="-1"></a>How does the Jaro-Winkler Distance work?</h2>
The Jaro-Winkler Distance computes a value between 0 and 1, where 0 indicates no similarity and 1 represents identical strings. It considers the number of matching characters, the number of transpositions (swapped characters), and a scaling factor for common prefix matches. The scaling factor in Jaro-Winkler gives higher weight to prefix similarities, making it especially effective for cases where slight misspellings or prefixes are common.
<h2 id="user-content-why-is-the-jaro-winkler-distance-important"><a href="#why-is-the-jaro-winkler-distance-important" aria-hidden="true" tabindex="-1"></a>Why is the Jaro-Winkler Distance important?</h2>
Jaro-Winkler is particularly useful when dealing with strings that might have minor discrepancies due to typos, abbreviations, or phonetic variations. Its emphasis on matching prefixes makes it robust against cases where two strings are similar but not identical. This makes it valuable for applications like deduplication, spell correction, and name matching.
<h2 id="user-content-how-is-the-jaro-winkler-distance-calculated"><a href="#how-is-the-jaro-winkler-distance-calculated" aria-hidden="true" tabindex="-1"></a>How is the Jaro-Winkler Distance calculated?</h2>
The calculation involves three main steps: computing the Jaro Distance (a value between 0 and 1), calculating the common prefix length, and applying the Jaro-Winkler formula. The formula takes into account the Jaro Distance, and the prefix scaling factor (usually 0.1 or 0.25), and adjusts the similarity score accordingly.
<h2 id="user-content-what-are-the-limitations-of-the-jaro-winkler-distance"><a href="#what-are-the-limitations-of-the-jaro-winkler-distance" aria-hidden="true" tabindex="-1"></a>What are the limitations of the Jaro-Winkler Distance?</h2>
While effective, the Jaro-Winkler Distance may not be ideal for all scenarios. It's sensitive to the choice of scaling factor, which needs to be tuned based on the specific dataset and application. Additionally, it might not capture more complex semantic differences between strings.
<h2 id="user-content-how-to-use-jaro-winkler-distance-to-find-duplicate-records"><a href="#how-to-use-jaro-winkler-distance-to-find-duplicate-records" aria-hidden="true" tabindex="-1"></a>How to use Jaro-Winkler Distance to find duplicate records?</h2>
The Jaro-Winkler distance is suited for <a href="/learn/data-cleaning/fuzzy-matching">Fuzzy Matching</a> and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.
<a href="/">Datablist</a> is a free <a href="/data-editor">data editor</a> with powerful <a href="/how-to/data-cleaning">data-cleaning features</a>. Datablist <a href="/features/duplicates-remover">Duplicates Finder</a> implements the Leventshtein distance to detect duplicate records across your datasets.
<img src="/learn_images/data-cleaning/jaro-winkler/jaro-winkler-dedupe.png#1384x1402" alt="Fuzzy Matching Algorithm">
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>


The Jaro-Winkler Distance is a similarity measure that quantifies the difference between two strings, often used in the context of record linkage, data deduplication, and string matching. It's an extension of the Jaro Distance metric and takes into account prefixes of matching characters.

## How does the Jaro-Winkler Distance work?

The Jaro-Winkler Distance computes a value between 0 and 1, where 0 indicates no similarity and 1 represents identical strings. It considers the number of matching characters, the number of transpositions (swapped characters), and a scaling factor for common prefix matches. The scaling factor in Jaro-Winkler gives higher weight to prefix similarities, making it especially effective for cases where slight misspellings or prefixes are common.

## Why is the Jaro-Winkler Distance important?

Jaro-Winkler is particularly useful when dealing with strings that might have minor discrepancies due to typos, abbreviations, or phonetic variations. Its emphasis on matching prefixes makes it robust against cases where two strings are similar but not identical. This makes it valuable for applications like deduplication, spell correction, and name matching.

## How is the Jaro-Winkler Distance calculated?

The calculation involves three main steps: computing the Jaro Distance (a value between 0 and 1), calculating the common prefix length, and applying the Jaro-Winkler formula. The formula takes into account the Jaro Distance, and the prefix scaling factor (usually 0.1 or 0.25), and adjusts the similarity score accordingly.

## What are the limitations of the Jaro-Winkler Distance?

While effective, the Jaro-Winkler Distance may not be ideal for all scenarios. It's sensitive to the choice of scaling factor, which needs to be tuned based on the specific dataset and application. Additionally, it might not capture more complex semantic differences between strings.

## How to use Jaro-Winkler Distance to find duplicate records?

The Jaro-Winkler distance is suited for [Fuzzy Matching](/learn/data-cleaning/fuzzy-matching) and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.

[Datablist](/) is a free [data editor](/data-editor) with powerful [data-cleaning features](/how-to/data-cleaning). Datablist [Duplicates Finder](/features/duplicates-remover) implements the Leventshtein distance to detect duplicate records across your datasets.

![Fuzzy Matching Algorithm](/learn_images/data-cleaning/jaro-winkler/jaro-winkler-dedupe.png#1384x1402)

[👉 Open Datablist (it's free)](https://app.datablist.com)



The Jaro-Winkler Distance is a similarity measure that quantifies the difference between two strings, often used in the context of record linkage, data deduplication, and string matching. It&amp#39;s an extension of the Jaro Distance metric and takes into account prefixes of matching characters.
How does the Jaro-Winkler Distance work?
The Jaro-Winkler Distance computes a value between 0 and 1, where 0 indicates no similarity and 1 represents identical strings. It considers the number of matching characters, the number of transpositions (swapped characters), and a scaling factor for common prefix matches. The scaling factor in Jaro-Winkler gives higher weight to prefix similarities, making it especially effective for cases where slight misspellings or prefixes are common.
Why is the Jaro-Winkler Distance important?
Jaro-Winkler is particularly useful when dealing with strings that might have minor discrepancies due to typos, abbreviations, or phonetic variations. Its emphasis on matching prefixes makes it robust against cases where two strings are similar but not identical. This makes it valuable for applications like deduplication, spell correction, and name matching.
How is the Jaro-Winkler Distance calculated?
The calculation involves three main steps: computing the Jaro Distance (a value between 0 and 1), calculating the common prefix length, and applying the Jaro-Winkler formula. The formula takes into account the Jaro Distance, and the prefix scaling factor (usually 0.1 or 0.25), and adjusts the similarity score accordingly.
What are the limitations of the Jaro-Winkler Distance?
While effective, the Jaro-Winkler Distance may not be ideal for all scenarios. It&amp#39;s sensitive to the choice of scaling factor, which needs to be tuned based on the specific dataset and application. Additionally, it might not capture more complex semantic differences between strings.
How to use Jaro-Winkler Distance to find duplicate records?
The Jaro-Winkler distance is suited for Fuzzy Matching and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.
Datablist is a free data editor with powerful data-cleaning features. Datablist Duplicates Finder implements the Leventshtein distance to detect duplicate records across your datasets.
Fuzzy Matching Algorithm
👉 Open Datablist (it&amp#39;s free)


What is the Jaro-Winkler Distance?

Levenshtein Distance, often referred to as edit distance, is a fundamental concept in string comparison and analysis. It quantifies the difference or similarity between two strings by calculating the minimum number of single-character edits required to transform one string into another. These edits can be in the form of insertions, deletions, or substitutions of individual characters.
<h2 id="user-content-how-does-levenshtein-distance-work"><a href="#how-does-levenshtein-distance-work" aria-hidden="true" tabindex="-1"></a>How does Levenshtein Distance work?</h2>
Imagine comparing the words "kitten" and "sitting." The Levenshtein Distance between these words is 3, as three operations are needed to change "kitten" into "sitting": substitute 'k' with 's,' insert 'i' after 'k,' and substitute 'e' with 'g.' This distance value provides a numerical measure of the similarity between the two words, considering variations like typos, misspellings, and other minor changes.
<h2 id="user-content-why-is-levenshtein-distance-important"><a href="#why-is-levenshtein-distance-important" aria-hidden="true" tabindex="-1"></a>Why is Levenshtein Distance important?</h2>
Levenshtein Distance has various practical applications. It serves as the basis for spell checkers, suggesting corrections for misspelled words. It's integral in fuzzy string matching, where data may have slight variations.
In bioinformatics, it helps analyze DNA sequences. By offering a systematic way to measure string differences, Levenshtein Distance plays a vital role in studying genetic variations and pattern recognition.
<h2 id="user-content-what-are-the-limitations-of-levenshtein-distance"><a href="#what-are-the-limitations-of-levenshtein-distance" aria-hidden="true" tabindex="-1"></a>What are the limitations of Levenshtein Distance?</h2>
While effective, Levenshtein Distance may not capture complex semantic differences between strings. Long strings can lead to larger computation times and memory usage. For more nuanced comparisons, other similarity metrics, like Jaccard similarity or cosine similarity, might be more suitable.
<h2 id="user-content-how-to-use-levenshtein-distance-to-find-duplicate-records"><a href="#how-to-use-levenshtein-distance-to-find-duplicate-records" aria-hidden="true" tabindex="-1"></a>How to use Levenshtein Distance to find duplicate records?</h2>
The Levenshtein distance is suited for <a href="/learn/data-cleaning/fuzzy-matching">Fuzzy Matching</a> and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.
<a href="/">Datablist</a> is a free <a href="/data-editor">data editor</a> with powerful <a href="/how-to/data-cleaning">data-cleaning features</a>. Datablist <a href="/features/duplicates-remover">Duplicates Finder</a> implements the Leventshtein distance to detect duplicate records across your datasets.
<img src="/learn_images/data-cleaning/excel-fuzzy-matching/excel-fuzzy-matching.png#1384x1402" alt="Fuzzy Matching Algorithm">
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>


Levenshtein Distance, often referred to as edit distance, is a fundamental concept in string comparison and analysis. It quantifies the difference or similarity between two strings by calculating the minimum number of single-character edits required to transform one string into another. These edits can be in the form of insertions, deletions, or substitutions of individual characters.

## How does Levenshtein Distance work?

Imagine comparing the words "kitten" and "sitting." The Levenshtein Distance between these words is 3, as three operations are needed to change "kitten" into "sitting": substitute 'k' with 's,' insert 'i' after 'k,' and substitute 'e' with 'g.' This distance value provides a numerical measure of the similarity between the two words, considering variations like typos, misspellings, and other minor changes.

## Why is Levenshtein Distance important?

Levenshtein Distance has various practical applications. It serves as the basis for spell checkers, suggesting corrections for misspelled words. It's integral in fuzzy string matching, where data may have slight variations.

In bioinformatics, it helps analyze DNA sequences. By offering a systematic way to measure string differences, Levenshtein Distance plays a vital role in studying genetic variations and pattern recognition.

## What are the limitations of Levenshtein Distance?

While effective, Levenshtein Distance may not capture complex semantic differences between strings. Long strings can lead to larger computation times and memory usage. For more nuanced comparisons, other similarity metrics, like Jaccard similarity or cosine similarity, might be more suitable.

## How to use Levenshtein Distance to find duplicate records?

The Levenshtein distance is suited for [Fuzzy Matching](/learn/data-cleaning/fuzzy-matching) and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.

[Datablist](/) is a free [data editor](/data-editor) with powerful [data-cleaning features](/how-to/data-cleaning). Datablist [Duplicates Finder](/features/duplicates-remover) implements the Leventshtein distance to detect duplicate records across your datasets.

![Fuzzy Matching Algorithm](/learn_images/data-cleaning/excel-fuzzy-matching/excel-fuzzy-matching.png#1384x1402)

[👉 Open Datablist (it's free)](https://app.datablist.com)



Levenshtein Distance, often referred to as edit distance, is a fundamental concept in string comparison and analysis. It quantifies the difference or similarity between two strings by calculating the minimum number of single-character edits required to transform one string into another. These edits can be in the form of insertions, deletions, or substitutions of individual characters.
How does Levenshtein Distance work?
Imagine comparing the words &ampquot;kitten&ampquot; and &ampquot;sitting.&ampquot; The Levenshtein Distance between these words is 3, as three operations are needed to change &ampquot;kitten&ampquot; into &ampquot;sitting&ampquot;: substitute &amp#39;k&amp#39; with &amp#39;s,&amp#39; insert &amp#39;i&amp#39; after &amp#39;k,&amp#39; and substitute &amp#39;e&amp#39; with &amp#39;g.&amp#39; This distance value provides a numerical measure of the similarity between the two words, considering variations like typos, misspellings, and other minor changes.
Why is Levenshtein Distance important?
Levenshtein Distance has various practical applications. It serves as the basis for spell checkers, suggesting corrections for misspelled words. It&amp#39;s integral in fuzzy string matching, where data may have slight variations.
In bioinformatics, it helps analyze DNA sequences. By offering a systematic way to measure string differences, Levenshtein Distance plays a vital role in studying genetic variations and pattern recognition.
What are the limitations of Levenshtein Distance?
While effective, Levenshtein Distance may not capture complex semantic differences between strings. Long strings can lead to larger computation times and memory usage. For more nuanced comparisons, other similarity metrics, like Jaccard similarity or cosine similarity, might be more suitable.
How to use Levenshtein Distance to find duplicate records?
The Levenshtein distance is suited for Fuzzy Matching and data deduplication. This algorithm can be used to compute similarity in names, addresses, or any text fields. Using a threshold value, records with a similarity distance above the threshold are considered as duplicate records.
Datablist is a free data editor with powerful data-cleaning features. Datablist Duplicates Finder implements the Leventshtein distance to detect duplicate records across your datasets.
Fuzzy Matching Algorithm
👉 Open Datablist (it&amp#39;s free)


What is the Levenshtein Distance?

Fuzzy matching is a sophisticated data comparison technique used to identify similarities between strings or records, even when they're not an exact match. Unlike traditional exact matching, which requires two strings to be identical to consider them a match, fuzzy matching takes into account variations, errors, and discrepancies in the data. This makes it highly effective in scenarios where data might be slightly different due to typos, abbreviations, misspellings, or other inconsistencies.
<h2 id="user-content-how-does-fuzzy-matching-work"><a href="#how-does-fuzzy-matching-work" aria-hidden="true" tabindex="-1"></a>How does Fuzzy Matching Work?</h2>
Fuzzy matching employs distance algorithms to compute a similarity score between two strings. This score quantifies how closely the strings resemble each other. Various methods are used to calculate this score, such as Levenshtein distance, Jaro-Winkler distance, and others. These algorithms analyze the characters, their sequence, and their positions to determine the level of similarity.
<h2 id="user-content-how-to-use-fuzzy-matching-for-data-deduplication"><a href="#how-to-use-fuzzy-matching-for-data-deduplication" aria-hidden="true" tabindex="-1"></a>How to use Fuzzy Matching for data deduplication?</h2>
Fuzzy matching is commonly used for data deduplication, where duplicate records in databases need to be identified and resolved.
<a href="/">Datablist</a> is a free <a href="/data-editor">data editor</a> with powerful <a href="/how-to/data-cleaning">data-cleaning</a> features. Datablist <a href="/features/duplicates-remover">Duplicates Finder</a> implements fuzzy matching algorithms to detect duplicate records across your datasets.
<a href="/how-to/remove-csv-duplicates">👉 Read our guide on CSV deduplication.</a>
<img src="/learn_images/data-cleaning/excel-fuzzy-matching/excel-fuzzy-matching.png#1384x1402" alt="Fuzzy Matching Algorithm">
<a href="https://app.datablist.com" target="_blank" rel="nofollow">👉 Open Datablist (it's free)</a>


Fuzzy matching is a sophisticated data comparison technique used to identify similarities between strings or records, even when they're not an exact match. Unlike traditional exact matching, which requires two strings to be identical to consider them a match, fuzzy matching takes into account variations, errors, and discrepancies in the data. This makes it highly effective in scenarios where data might be slightly different due to typos, abbreviations, misspellings, or other inconsistencies.

## How does Fuzzy Matching Work?

Fuzzy matching employs distance algorithms to compute a similarity score between two strings. This score quantifies how closely the strings resemble each other. Various methods are used to calculate this score, such as Levenshtein distance, Jaro-Winkler distance, and others. These algorithms analyze the characters, their sequence, and their positions to determine the level of similarity.

## How to use Fuzzy Matching for data deduplication?

Fuzzy matching is commonly used for data deduplication, where duplicate records in databases need to be identified and resolved. 

[Datablist](/) is a free [data editor](/data-editor) with powerful [data-cleaning](/how-to/data-cleaning) features. Datablist [Duplicates Finder](/features/duplicates-remover) implements fuzzy matching algorithms to detect duplicate records across your datasets.

[👉 Read our guide on CSV deduplication.](/how-to/remove-csv-duplicates)

![Fuzzy Matching Algorithm](/learn_images/data-cleaning/excel-fuzzy-matching/excel-fuzzy-matching.png#1384x1402)

[👉 Open Datablist (it's free)](https://app.datablist.com)





Fuzzy matching is a sophisticated data comparison technique used to identify similarities between strings or records, even when they&amp#39;re not an exact match. Unlike traditional exact matching, which requires two strings to be identical to consider them a match, fuzzy matching takes into account variations, errors, and discrepancies in the data. This makes it highly effective in scenarios where data might be slightly different due to typos, abbreviations, misspellings, or other inconsistencies.
How does Fuzzy Matching Work?
Fuzzy matching employs distance algorithms to compute a similarity score between two strings. This score quantifies how closely the strings resemble each other. Various methods are used to calculate this score, such as Levenshtein distance, Jaro-Winkler distance, and others. These algorithms analyze the characters, their sequence, and their positions to determine the level of similarity.
How to use Fuzzy Matching for data deduplication?
Fuzzy matching is commonly used for data deduplication, where duplicate records in databases need to be identified and resolved. 
Datablist is a free data editor with powerful data-cleaning features. Datablist Duplicates Finder implements fuzzy matching algorithms to detect duplicate records across your datasets.
👉 Read our guide on CSV deduplication.
Fuzzy Matching Algorithm
👉 Open Datablist (it&amp#39;s free)


Knowledge Center

CSV Manipulation

How to count characters in a CSV column?

How to count distinct values in a CSV column?

Do CSV files need headers?

What does CSV stand for?

What is a CSV file used for?

What does Dedupe mean?

What is the difference between join and merge?

Download Sample CSV Files for free

Get the number of rows of a CSV file on Linux or macOS Terminal

How do I open a .csv file?

How to reorder and/or remove CSV columns online?

How to search in a big CSV file using Linux or macOS Terminal?

How to split a big CSV file using Linux or macOS Terminal?

Get the Sum, Min, Max, and Average values of a CSV column

How to count words in a CSV Column?

Spreadsheet

What is Apple Numbers row limit?

How to convert an Apple Numbers file to CSV?

What is the difference between a spreadsheet and a database?

How to export selected cells from Excel into a CSV file

How to convert an XLSX file to CSV with Microsoft Excel?

Should I use a spreadsheet or a database?

Marketing

What is lead deduplication?

Scraping

How to extract all URLs from a webpage?

Data Cleaning

What is deduplication?

What is the Double Metaphone algorithm?

What is the Jaro-Winkler Distance?

What is the Levenshtein Distance?

What is Fuzzy Matching?