Is there a way load a GCS file instead of first downloading it and then opening it in your programming language

Question

I've a ML-model stored on a Google Storage Bucket, but now I'm having some trouble using it.
I thought instead of first downloading the model (1.25GB) I could just direcly load the data into the memory, but I couldn't really find any documentation on this.

So I want to instantly get file data into a variable, is this possible?

Answer 1

In addition, you could use Streaming Transfers.

Cloud Storage supports streaming transfers, which allow you to stream data to and from your Cloud Storage account without requiring that the data first be saved to a file. Streaming transfers are useful when you want to upload data generated from a process directly into Cloud Storage, or when you want to download data from Cloud Storage into a process.

As explained in the documentation, you can use the third party boto client library plugin for Cloud Storage to utilize this tool.

Answer 2

I solved by usign the download_as_string function (Python only).
https://googleapis.dev/python/storage/latest/_modules/google/cloud/storage/blob.html#Blob.download_as_string
But this requires to convert the data from bytes into the preferred type

blob = bucket.blob(storage_name)
x = blob.download_as_string()
print(x.decode('utf-8'))

In Java you can probably use this:
How to read a file from Google Cloud Storage in Java
but I haven't tried it.