How do you use the Cassandra tool sstableloader?

Question

I'm trying to use the sstableloader to load data into an existing Cassandra ring, but cant figure out how to actually get it to work. I'm trying to run it on a machine that has a running cassandra node on it, but when I run it I get an error saying that port 7000 is already in use, which is the port the running Cassandra node is using for gossip.

So does that mean I can only use sstableloader on a machine that is in the same network as the target cassandra ring, but isn't actually running a cassandra node?

Any details would be useful, thanks.

As you point out in your answer (and tutorial), you can be on the same node, you just have to provide alternate ipaddress. But, there is a simpler way; you can use the JMX->StorageService->bulkload() call. Works great for testing and occasional use. — libjack, Nov 17 '11 at 22:21

score 5 · Accepted Answer · answered Jul 26 '11 at 20:00

5

Played around with sstableloader, read the source code, and finally figured out how to run sstableloader on the same machine that hosts a running cassandra node. There are two key points to get this running. First you need to create a copy of the cassandra install folder for sstableloader. This is becase sstableloader reads the yaml file to figure out what ipaddress to use for gossip, and the existing yaml file is being used by Cassandra. The second point is that you'll need to create a new loopback ipaddress (something like 127.0.0.2) on your machine. Once this is done, change the yaml file in the copied Cassandra install folder to listen to this ipaddress.

I wrote a tutorial going more into detail about how to do this here: http://geekswithblogs.net/johnsPerfBlog/archive/2011/07/26/how-to-use-cassandrs-sstableloader.aspx

answered Jul 26 '11 at 20:00

Turbo

2,490
4
25
30

@Turbo Broken link. Add any other missing bits here? – Kasisnu Mar 03 '15 at 09:54
4

@Turbo broken link :( – Don Branson Jul 28 '15 at 00:52
This is similar and might be helpful :- http://amilaparanawithana.blogspot.in/2012/06/bulk-loading-external-data-to-cassandra.html – AlphaBetaGamma May 30 '16 at 08:20
1

And this is **EXACTLY** why links to answers are not answers according to the Stack Exchange guidelines! – Bruno Bronosky Apr 11 '18 at 16:24
Web archive link for the article mentioned in the answer: https://web.archive.org/web/20131031112940/http://geekswithblogs.net/johnsPerfBlog/archive/2011/07/26/how-to-use-cassandrs-sstableloader.aspx – yildirim Jul 26 '23 at 11:39

score 1 · Answer 2 · answered Jul 28 '11 at 23:29

1

The Austin Cassandra Users Group just had a presentation on this: http://www.slideshare.net/alex_araujo/etl-with-cassandra-streaming-bulk-loading/

answered Jul 28 '11 at 23:29

zznate

1,898
13
12

score 0 · Answer 3 · answered Jul 10 '14 at 14:17

0

For Reference SEE: Using Cassandra SStableloader for bulkloading the data into cassandra http://ramuprograms.blogspot.com/2014/07/bulk-loading-data-into-cassandra-using.html

answered Jul 10 '14 at 14:17

Venkata Ramu Kandulapati

21
6

score 0 · Answer 4 · answered Oct 08 '16 at 00:33

If you are looking to do this in Java see below utility class:

BulkWriterLoader

    List<String> argList = new ArrayList<>();
    argList.add("-v");
    argList.add("-d");
    argList.add(params.hosts);
    argList.add("-f");
    argList.add(params.cassYaml);
    argList.add(params.fullpath);
    LoaderOptions options = LoaderOptions.builder()
            .parseArgs(argList.stream().toArray(String[]::new))
            .build();
    try
    {
        BulkLoader.load(options);
    }
    catch (BulkLoadException e)
    {
        e.printStackTrace();
    }
    ...

The code will also generate the sstable files using the CQLSSTableWriter class.

score 0 · Answer 5 · answered Dec 20 '19 at 14:14

Things improve and the whole procedure of using sstableloader is much easier including a easier way to generate sstables with CQLSSTableWriter.

For all the details: https://docs.datastax.com/en/archived/cassandra/3.0/cassandra/tools/toolsBulkloader.html

samarth · Answer 6 · 2012-03-12T10:18:25.777

I have used the sstableloader utility provided in cassandra-0.8.4 to successfully load the sstables into cassandra.From Some of the issues i have faced i have following tips

If you are running it on single machine,you have to create a copy the cassandra installation folder and have to run sstable-loader from this folder.Also change the listen address,rpc address also provide the ip address of running cassandra as seeds in cassandra.yaml file of this copied one.Check if the cluster name in both the cassandra.yaml file is same.
These sstables have to be in a directory whose name is the name of the keyspace
It requires a directory containing a cassandra.yaml configuration file in the classpath.
Note that the schema for the column families to be loaded should be defined beforehand

For Reference SEE: Using Cassandra SStableloader

Does sstableloader load just part of the data? I have cluster1 with 3 nodes and cluster2 with 3 nodes. I need to copy data from cluster1 to cluster2. Do I have to run sstableloader on each cluster1 node or sstableloader does get all parts of table from other hosts for me? — Capacytron, Mar 31 '15 at 10:11

How do you use the Cassandra tool sstableloader?

6 Answers6

Linked