How to read a large CSV file in Node.js
Today, we will show you how to read a large CSV file in Node.js. In this article, we will create an example to read a large CSV file using `csv-parse`.
Checkout more articles on Node.js
- Confirm a stripe paymentIntent using Node.js
- Send an email with attachment using Node.js
- How to enable CORS for multiple domains in Node.js
- File Upload in Node.js
Here, we will use the previously generated large CSV file. Refer the following article for more details.
How to write a large CSV file in Node.js
Solution
Use the following code to read the large CSV file using Node.js. Here we have used the `fs.createReadStream` along with the csv-parse npm package to read a CSV file.
var fs = require('fs');
var { parse } = require('csv-parse');
// ...
// ...
const fileName = "product_10L.csv";
fs.createReadStream(fileName)
.pipe(parse({ delimiter: ",", from_line: 1 }))
.on("data", function (row) {
console.log('Row: ', row);
}).on("end", function () {
console.log('Process Completed.');
}).on("error", function (error) {
console.error(error.message);
});
In the above code, we are trying to read product CSV file that contains the product name and product SKU. You can process the data immediately after reading.
API Example
Your `server.js` file should look like below if you add this logic to the rest API. In this code we read the CSV file from the root level.
Refer this article to setup REST API in node.js.
Create REST API in Node.js
server.js
var express = require('express'),
app = express(),
port = process.env.PORT || 4000;
var fs = require('fs');
var { parse } = require('csv-parse');
app.get('/read-csv', async (req, res) => {
const fileName = "product_10L.csv";
fs.createReadStream(fileName)
.pipe(parse({ delimiter: ",", from_line: 1 }))
.on("data", function (row) {
console.log('Row: ', row);
}).on("end", function () {
console.log('Process Completed.');
res.send({ success: true });
}).on("error", function (error) {
console.error(error.message);
res.send({ error: true });
});
});
app.listen(port, () => {
console.log('Server started on: ' + port);
});
Here, we have logged the row data on `data` listener. Also, we returned `success: true` message at the end of the process and added the listener to catch the error.
Run the following command to read the CSV.
http://localhost:4000/read-csv
That’s it for today.
Thank you for reading. Happy Coding..!! 🙂