Quality blog content since 2006
- the imjtk.com successor
Quality blog content since 2006
- the imjtk.com successor

Blog Post

Importing JSON in Firebase Firestore

August 16, 2020 Dev, javascript
Importing JSON in Firebase Firestore

If you know anything about me you know that I love the Google Cloud Platform and Firebase in particular.

Google really has nailed a system for publishing a web app, and with Firebase functions you can write Node.js on the back end to create a full stack as it were.

In life, however, nothing is perfect. There are a number of issues that I have with Firestore in particular. One of them is that there is no good, simple data import / export. Coming from an SQL background this seems like a step backwards.

But in a world when everything is config and code I started thinking of how one might use that paradigm to populate a Firestore collection with JSON data. I of coursed searched stack overflow and the Google documentation. As is often the case Stack Overflow was the more helpful of the two…

The trick to getting this to work is getting an NPM called “firestore-export-import“. Create a directory and do the npm init and npm install deal.

The solution to the other half of this problem comes in the form of a couple of config files from Firebase and like seven lines of JavaScript.

The first file you’ll need you can get from the Firebase SDK snippet. Go to Project Overview and choose Project settings. Scroll down to the Web apps section and under the Firebase SDK click on the Config radio button. Copy and paste that code into a file called config.js in you project directory. At the bottom, under your newly pasted firebaseConfig object paste this one line:

module.exports = firebaseConfig;

and save the file.

Next you need to create another file called service-account.json This is where StackOverflow came into play for me. I use the firebase config to init my projects, but this service account deal was a new one for me. Essentially you can visit:

https://console.firebase.google.com/project/**YOUR_PROJECT**/settings/serviceaccounts/adminsdk

Click the Generate New Private Key button at the bottom of the Firebase Admin SDK section of the Service Accounts tab.

This file should never be uploaded to public repositories and if compromised can leave your project 100% vulnerable to any number of nefarious activities.

We now we have a directory with our package.json, node_modules, and two config files. This would be a good time to copy the JSON file you want to import into Firebase into this directory as well. I scraped Twitter for a bunch of tweets about ETH – so as you can see I have an eth.json file in my directory listing below.

You might have also noticed that I have one additional file that we have not gone over yet as well – and that is the file that pulls this all together firebase-import.js.

My entire import script is only 25 lines long, but you really only need 7 lines. First you need the to import your config files and the firestore-export-import npm.

const firestoreService = require('firestore-export-import');
const config = require('./config.js');
const account = require('./service-account.json');

And then make a variable and since it will take some time give it the async and await treatment:

const importJson = async () => {
  try {
    await firestoreService.initializeApp(account, config.databaseURL);
    await firestoreService.restore('./eth.json');
  }catch (err) {console.log(err);}
};

importJson();

Of course you will want to replace the eth.json file name in the example with the file name of your JSON file. You should then be able to run:

node firebase-import.js

to upload your local JSON data to Firebase Firestore.

So what do you think – does that make sense? Did you try it? Did it work for you? I just re-ran it to make the feature image for this post and it worked for me. Feel free post a comment if you are having difficulty getting this to work or if you have any questions, I’d be happy to help.

Taggs:
5 Comments
  • Ryan 9:43 pm November 15, 2020 Reply

    Google has now an official way to export your data although it is very convoluted with a storage bucket etc. I think your method is better 🙂

  • Jason G 10:18 am November 22, 2020 Reply

    Thanks for posting this – but how do I export my Firebase data? I need to move it to MongoDB.

  • Nesha 12:31 pm November 29, 2020 Reply

    I can’t get this to work! Do you have a git repository?

    • JTk 4:25 pm December 4, 2020 Reply

      Hey Now – there is, here:

      https://github.com/TripKendall/FirebaseJsonUpload

      Keep in mind that you will need to add your firebase info to the two config files. Also in the import.js file itself I have the file hardcoded as “btc-twitter.json” which I have included – but you’ll want to replace that with the path / name of your json file.

  • WillFord 10:45 am December 27, 2020 Reply

    Thanks for posting, this got me started and then I figured it out. I did not get it to work exactly as you described but I did get it to work.

Write a comment