How to read a file from Google Cloud Storage in Java

Question

I would like to read a file from Google Cloud storage using Java. The below link was not helpful as I dont use HttpServletRequest and HttpServletResponse.

Reading in a file from google cloud storage using java

Is there any other way by which I can accomplish this? I m writing a simple stand alone program as POC

score 16 · Answer 1 · edited Jul 05 '18 at 04:47

16

The simplest way to accomplish this is to use Google's google-cloud Java library. Downloading will look something like this:

String PROJECT_ID = "my-project";
String PATH_TO_JSON_KEY = "/path/to/json/key";
String BUCKET_NAME = "my-bucket";
String OBJECT_NAME = "my-object";

StorageOptions options = StorageOptions.newBuilder()
            .setProjectId(PROJECT_ID)
            .setCredentials(GoogleCredentials.fromStream(
                    new FileInputStream(PATH_TO_JSON_KEY))).build();

Storage storage = options.getService();
Blob blob = storage.get(BUCKET_NAME, OBJECT_NAME);
ReadChannel r = blob.reader();

edited Jul 05 '18 at 04:47

Personius

79
10

answered May 22 '17 at 21:02

Brandon Yarbrough

37,021
23
116
145

THanks Brandon, Actually I am very new to the whole concept. In this case I am not sure what are PROJECT_ID, PATH_TO_JSON_KEY, OBJECT_NAME etc (Bucket name i know !). Please help – sreejithpin May 23 '17 at 12:13
Whenever you are working in Google Cloud Storage, you are working within a "project". It is the first thing you create and represents a group of work across several cloud tools that are all billed the same way. It has a string identifier; you'll see it on your homepage of the cloud console. – Brandon Yarbrough May 23 '17 at 21:45
A standalone program running Java code needs some sort of authorization to read objects. This is usually done by creating a service account in a project that represents that program. The service account will come with a JSON file that can be used by libraries to authenticate as that service account. PATH_TO_JSON_KEY is the location in your local filesystem of that file. – Brandon Yarbrough May 23 '17 at 21:46
GCS buckets have globally unique names, like "my-gcs-bucket-75". BUCKET_NAME is the name of the bucket from which you're reading. GCS objects also have names, perhaps like "images/foo.jpeg". That's what OBJECT_NAME is. – Brandon Yarbrough May 23 '17 at 21:47
You may find the tutorials here useful: https://cloud.google.com/getting-started/ – Brandon Yarbrough May 23 '17 at 21:47
@BrandonYarbrough It support only for Java 8 right? For Java 7 what is the other option we can use? Can you please help me as I am using Java 7 – Harshal_Kalavadiya Dec 07 '18 at 07:07

score 8 · Answer 2 · answered May 26 '17 at 14:17

8

Read this GCloud doc for more info.

Your code should be:

Storage storage = StorageOptions.newBuilder()
            .setProjectId(projectId)
            .setCredentials(GoogleCredentials.fromStream(new FileInputStream(serviceAccountJSON)))
            .build()
            .getService();
Blob blob = storage.get(BUCKET_URL, OBJECT_URL);
String fileContent = new String(blob.getContent());

answered May 26 '17 at 14:17

2

note this reads the file fully in the memory, which might be not desired if the file is huge – Vojtěch Nov 01 '18 at 23:00
yeah, if you need to write the file to disk. -> https://stackoverflow.com/a/44205060/6259970 – Nov 21 '18 at 11:18

score 0 · Answer 3 · answered Nov 29 '22 at 18:46

Read from the Storage bucket File

GoogleCredentials credentials  = GoogleCredentials.fromStream(
     new FileInputStream(CREDENTIAL_PATH));    Storage storage =
 StorageOptions.newBuilder()
.setCredentials(credentials)
.setProjectId(PROJECT_ID).build().getService();

String fileData = new String(storage.get(BUCKET_NAME, FILE_NAME), StandardCharsets.UTF_8);

Read from Directory - Files If Storage bucket contains directories and directories contains Files

Storage Bucket Object same as earlier

Page<Blob> list = storage.list(BUCKET_NAME, 
Storage.BlobListOption.prefix(DIRECTORY_NAME));

for (Blob blob: list.getValues()) {
    // Read all files
    String fileData =  new String(blob.getContent());
    System.out.println(fileData);
}

Note - it will read all data in memory. So if file size is huge(in GBs) , not recommend to read in memory files.

How to read a file from Google Cloud Storage in Java

3 Answers3

Linked