Accessing Google Big Query from Jupyter notebook.

Working with Big Query datasets within Jupyter notebook is no different to work with any other DB using Jupyter. In 3 steps we can query data stored in BQ from Jupyter notebook.

Step#1

Before we read or query data, we need to make sure to meet few prerequisites.

  1. You should have an active google cloud account (BQ Sandbox works too). You can use an existing project or create new project.
  2. You should have Jupyter notebook installed on your computer.
  3. You should enable Big query API either from cloud console or from Cloud Shell.

To enable Big Query API through cloud console: Go to navigation menu -> click APIs & Services. Once you are there, click + Enable APIs and Services (Highlighted below). In search bar, enter BigQuery API and click Enable.

Alternatively you can activate the API from cloud shell by using the below command.

Step#2

In order to make request to the API enabled in Step#1, you need to create a service account and get an authentication file for your Jupyter Notebook. To do so, navigate to Credentials tab under APIs and Services console and click Create Credentials tab and then Service account under drop down.

Enter Service account name and description. You can use BigQuery Admin role under Grant this service account access to project. Click Done. You can now see the new service account under Credentials screen. Click pencil icon beside the new service account you have created and click Add Key to add auth key. Please choose JSON and click CREATE. It will download the JSON file with auth key info. (Download path will be used to authenticate)

Step#3

You are now all set to use big query within Jupyter notebook. Before starting to query data, install the following from command prompt.

Querying public dataset (bigquery-public-data.austin_crime.crime) to get the top crimes by year:

Data analysts who work with Jupyter notebook day in day out can leverage this form of querying data which helps them to be more productive.

Please note this is one of many ways to query bigquery data from Jupyter.

References:

Lead Programmer at Novartis Healthcare Pvt Ltd.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store