Importing data from multiple csv files in D3

Question

I am brand new to D3 and just started working on an a project. My question is this. I want to import data from 2 csv files in D3 to use them for graph comparisons. The problems I am facing are:

1.How do I import data from multiple csv files.
2.Can I use one array for each csv or does D3 use only one global data array?
3.Is there a way to choose a certain column from the csv files to import?

Here is an example, I want to import the "oldVer" from each of the files in separate arrays and then use the 2 arrays to work with. Is that posible in D3 and how?

csv 1
time,oldVer,newVer,oldT,newT
1,180930,190394,24,59
2,198039,159094,26,45
3,152581,194032,22,61

csv 2
time,oldVer,newVer,oldT,newT
1,184950,180435,27,26
2,120590,129409,13,13
3,165222,182133,60,54

Again sorry for the dumb question but I have found little feedback on this matter. Any help will be appreciated.

score 37 · Answer 1 · answered Jun 30 '18 at 08:48

37

In d3 version 5, you can use Promise.all to load multiple csv files. Example:

Promise.all([
    d3.csv("file1.csv"),
    d3.csv("file2.csv"),
]).then(function(files) {
    // files[0] will contain file1.csv
    // files[1] will contain file2.csv
}).catch(function(err) {
    // handle error here
})

More info about loading csv in d3 v5

More info about Promise.all()

answered Jun 30 '18 at 08:48

Abang F.

876
8
12

1

Thanks @Abang F, the simplest and clearest demonstration I've seen :) – Markus Jul 11 '18 at 07:57
Can I use this somehow with a dynamic number of files? – timbmg Oct 23 '18 at 10:22
1

@blckbird `Promise.all(fileList.map(d3.csv)).then(...)` – BallpointBen Jun 12 '20 at 04:33

score 17 · Answer 2 · answered Feb 18 '14 at 01:27

17

You simply call d3.csv several times:

d3.csv("csv1.csv", function(error1, data1) {
  d3.csv("csv2.csv", function(error2, data2) {
    // do something with the data
  });
});

As for your third question, no, D3 will parse everything. There's nothing forcing you to use all the data though, so if you're interested in only one column, just use the data from that.

answered Feb 18 '14 at 01:27

Lars Kotthoff

107,425
16
204
204

1

Hi Lars, since the objective is to compare the data and thus have both available at the same time, there would be a need to control the flow of execution of these nested asynchronous calls, no? If we do something inside the inner function, there is no guarantee that data1 will be ready. Am I right? I was reading [this](http://book.mixu.net/node/ch7.html). It is talking about node.js but perhaps it would apply. – FernOfTheAndes Feb 18 '14 at 02:39
Right on spot, I will try it out now. I've read about asynchronous calls before, but cannot say much, I just have to try it out. Thank you for the quick responce guys. – John M. Feb 18 '14 at 11:14
IF I am right here, in the link I referenced, I believe you want 7.2.2 Control flow pattern #2, where "...we just want to take a small set of operations, launch them all in parallel and then do something when all of them are complete." – FernOfTheAndes Feb 18 '14 at 12:11
1

@FernOfTheAndes No -- this is exactly what the nesting takes care of. I'm sending the request for the second CSV only when the first request is complete, so inside the second handler function *both* are guaranteed to be available. – Lars Kotthoff Feb 18 '14 at 13:53
I see now...the inner call will only run when the data returns from the outer call. Sorry for the confusion. – FernOfTheAndes Feb 18 '14 at 14:15
Guys I've been away for a long while after I did this post. I'm more familiar with things now and see how this could apply. Will try it in a few days, hope it works. Didn't expect nesting to work, but seems logical. Again I will respond with feedback. Thanks again for the responds and sorry for the late response by me. – John M. Jun 27 '15 at 14:45
@LarsKotthoff I finally got around by using the queue.js. Worked like a charm. Problems with other stuff still coming up, but I have the working code on another question I did about tooltips in case anyone is interested of its use. – John M. Jul 13 '15 at 16:38

Stuart Hallows · Answer 3 · 2016-09-14T06:45:13.003

15

You could use a d3 queue to load the files simultaneously. An example;

d3.queue()
.defer(d3.csv, "file1.csv")
.defer(d3.csv, "file2.csv")
.await(function(error, file1, file2) {
    if (error) {
        console.error('Oh dear, something went wrong: ' + error);
    }
    else {
        doStuff(file1, file2);
    }
});

edited Sep 14 '16 at 06:45

answered Sep 14 '16 at 06:40

Stuart Hallows

8,795
5
45
57

1

does this work for json as well? e.g. ``` d3.queue()/* to load multiple datasets */ .defer(d3.json, "data1") .defer(d3.json, "data2") .await(function(error, data1, data2) { if (error) { console.log('Something went wrong: ' + error); } else { //stuff for graph } });``` – thesayhey Feb 23 '17 at 22:42
Great way of doing that! – Aug 02 '17 at 06:46

score 1 · Answer 4 · edited Mar 19 '18 at 20:40

1

To answer your part 3,

Is there a way to choose a certain column from the csv files to import?

No, you cannot load in part of a CSV. You can, however, load in the entire CSV file and selectively use one column from it. You can refer to data.newVer to utilize the newVer column data.

edited Mar 19 '18 at 20:40

FrankS101

2,112
6
26
40

answered Mar 19 '18 at 19:05

travelsize

74
1
4

Importing data from multiple csv files in D3

4 Answers4

Linked