Reverse Geocode Zip 4

With ZIP+4 geocoding, the GEOCODE procedure attempts to match the five-digit ZIP code and ZIP+4 extension from your address data set with the lookup data set. If a ZIP+4 code is not found, the GEOCODE procedure attempts to match the standard five-digit ZIP code. With ZIP+4 geocoding, the GEOCODE procedure attempts to match the five-digit ZIP code and ZIP+4 extension from your address data set with the lookup data set. If a ZIP+4 code is not found, the GEOCODE procedure attempts to match the standard five-digit ZIP code.

• Reverse Geocode Zip 4 Free
• Reverse Geocode Zip 4 Code
• Reverse Geocoding Zip 4

This tutorial shows how to use Google Cloud Platform (GCP) to build anapp that receives telemetric data about geolocation, processes it, and thenstores the processed and transformed data for further analysis.

This tutorial illustrates the concepts shown inBuilding a Scalable Geolocation Telemetry System Using the Maps API.It includes and uses data from theSan Diego, California, freeways public dataset.This data was captured from actual automobile journeys, recorded byusing road sensors. You canregister for an accountto have access to the full dataset if you want to run your ownexperiments with additional data.

• Perform reverse geocoding locally and offline. Contribute to AReallyGoodName/OfflineReverseGeocode development by creating an account on GitHub.
• Geocoding And Reverse Geocoding. In the tutorial below, I use pygeocoder, a wrapper for Google’s geo-API, to both geocode and reverse geocode.

Note: This version of the tutorial contains the full steps to run the codemanually. To try a version that uses Docker and Cloud Shell, seethe GitHub readme.

The tutorial:

• Starts with traffic data stored in CSV files.
• Processes messages in a Cloud Pub/Sub queue.
• Reverse geocodes latitude and longitude to convert the coordinates to a street address.
• Calculates the elevation above sea level.
• Converts from Coordinated Universal Time (UTC) to local time by querying which timezone each location is in.
• Writes the data, with the added geographic contextual information, to a BigQuery dataset for your analysis.
• Visualizes the data as heat maps superimposed over a map of the San Diego metro area.

The following diagram illustrates the major components and how the data movesbetween them:

Objectives

• Creating credentials.
• Setting up Cloud Pub/Sub and BigQuery.
• Loading and analyzing the data in BigQuery.
• Visualizing the data on a web page.
• Understanding the code.

Costs

This tutorial uses billable components of GCP, including:

• BigQuery (5 GB storage, 5 GB streaming inserts)
• Cloud Pub/Sub (< 200K operations)
• Google Maps Platform

The cost of running this tutorial will vary depending on run time. Use thepricing calculator estimate to see a cost estimate based on yourprojected usage.

New GCP users might beeligible for a free trial.

The Maps API standard plan offers a free quota and pay-as-you-go billing afterthe quota has been exceeded. If you have an existing license for the Maps API orhave a Maps APIs Premium Plan,see the documentation firstfor some important notes. You can purchase a Maps APIs Premium Plan for higherquotas.

You must have a Google Maps license for any app that restricts access,such as behind a firewall or on a corporate intranet. For more details aboutGoogle Maps Platform pricing and plans, seethe online documentation.

Before you begin

Set up GCP

1. Select or create a Google Cloud Platform Console project.

2. Enable billing for your project.

3. Click the following button to enable the required GCP APIs. If prompted, be sure to select the project you created in step 1.

These APIs include:

Reverse Geocode Zip 4 Free

• BigQuery API
• Cloud Pub/Sub API
• Cloud Storage
• Google Maps Geocoding API
• Google Maps Elevation API
• Google Maps Time Zone API
• Google Maps JavaScript API

Set up your development environment

1. Install the Cloud SDK.

2. Initialize and authenticate the gcloud command-line tool:

3. Set up your Python development environment. Follow these instructions to install on Linux, macOS, or Windows.

   On Debian/Ubuntu, be sure to install build-essential:

4. Download the code and other files to your local computer. To clone the GitHub repository by using git:

   Alternatively, you candownload and unzip the archive.

   Note: The remainder of this tutorial assumes that the root directoryfor the source code is named bigquery-reverse-geolocation.

5. For Debian-based systems, install additional prerequisites:

   1. Change to the resources directory from the directory where you downloaded the code:

   2. If you don't have it already, install pip:

   3. Install the prerequisites:

   4. Change directory back to the root of the project:

Creating credentials

For this tutorial, you'll need the following credentials:

• An OAuth 2.0 client ID.
• A Google API server key.
• A Google API browser key.

Important: Keep your API keys secure. Publicly exposing your credentialscan result in your account being compromised, which could lead to unexpectedcharges on your account. To keep your API keys secure,follow these best practices.

Creating an OAuth 2.0 client ID

Create a client ID that you can use to authenticate user requests toBigQuery. Follow these steps:

1. Click the following button to open the Credentials page in the GCP Console. If you have more than one project, you might be prompted to select a project.

2. Click Create credentials > OAuth client ID.

3. Select Web application.

4. Enter 'Maps API client ID' in the Name field.

5. Add the following two origin URLs in the Restrictions section, in Authorized JavaScript origins and Authorized redirect URIs

   Adding these URLs enables an user to access BigQuery datathrough JavaScript running in a browser. You need this authorization for anupcoming section of the tutorial, when you display a visualization of dataon a map in your web browser.

6. Click Create to generate the new client ID.

7. Click OK to close the dialog box that shows your new client ID and secret.

Creating a server key

1. In the GCP Console, click Create credentials > API key.

2. Click Restrict key.

3. Name the key Maps tutorial server key.

4. In the Key restriction section, select IP addresses.

5. In the Accept requests from these server IP addresses field, enter the IPv4 address of you computer, which you noted in the previous section.

6. Click Save.

7. Stay on the page.

Creating a browser key

The browser key is a requirement for using the Maps JavaScript API.

1. Click Create credentials > API key.

2. Click Restrict key.

3. Name the key Maps tutorial browser key.

4. Click Save.

Setting up Cloud Pub/Sub

Cloud Pub/Sub is the messaging queue that handles moving the data fromCSV files to BigQuery. You need to create a topic, whichpublishes the messages, and a subscription, which receives thepublished messages.

Create a Cloud Pub/Sub topic

The topic publishes the messages. Follow these steps to create the topic:

1. Browse to the Cloud Pub/Sub topic list page in the GCP Console:

2. Click Create a topic. A dialog box opens.

3. Add traffic to the end of the path that is provided for you in the Name field. The path is determined by the system. You can provide only the name for the topic.

4. Click Create. The dialog box closes.

5. Stay on the page.

Create a Cloud Pub/Sub subscription

The subscription receives the published messages. Follow these steps to createthe subscription:

1. In the topic list, in the row that contains the traffic topic, click the downward arrow on the right-hand end of the row.

2. Click New subscription to open the Create a new subscription page.

3. Add mysubscription to the end of the path that is provided for you in the Subscription name field.

4. Select Pull for the Delivery Type.

5. Click Create.

Setting up BigQuery

To prepare BigQuery to receive the data that you want to analyze,you must create a dataset, which is the logical container for the data tables,and then add a table to the dataset, which specifies the schema and stores thedata in the specified format.

1. In your terminal, change to the resources directory:

2. Create the empty dataset, named sandiego_freeways:

   The terminal shows a confirmation message:

3. Add the table, named geocoded_journeys, to the dataset:

   The terminal prints a confirmation message:

   This command creates a table that conforms to the schema defined in geocoded_journeys.json.

Viewing the schema

To view the schema in the BigQuery console, follow these steps:

1. Open the BigQuery Console:

2. Near the top of the left-hand panel, locate your project name. You should see sandiego_freeways listed below the project name.

3. Click sandiego_freeways to expand the node.

4. Click geocoded_journeys.

You should see the schema displayed in the right-hand panel of the console:

Loading the data into BigQuery

Now you can perform the steps that import the data from the CSV files, transcodethe data, and load it into a BigQuery table.

Pushing the data to the topic

To push the data to the topic, first modify the setup file and then run thePython script.

Modifying the setup file

1. From the resources directory, use your preferred text editor to edit setup.yaml.

2. For PROJECT_ID, replace your-project-id with your project ID. Keep the single quotation marks in this and all other values that you replace.

3. For DATASET_ID, don't change sandiego_freeways.

4. For TABLE_ID, don't change geocoded_journeys.

5. For PUBSUB_TOPIC, replace your-project-id with your project ID.

6. For ROOTDIR, replace the provided path with resources/data.

7. For SUBSCRIPTION, replace your-project-id with your project ID.

8. For MAPS_API_KEY, replace Your-server-key with the server key you created and named 'Maps tutorial server key'. You can see your credentials by clicking the following button:

9. Save and close the file.

Running the push script

From the bigquery-reverse-geolocation root directory, run the Python scriptthat populates the Cloud Pub/Sub topic.

1. Change directory to the root of the project:

2. Run the script:

You should see repeated lines of output like this one:

This output confirms that each line of data has been pushed. If you don't seemany lines of such output, double-check setup.yaml to ensure that you providedall the correct path and name information.

It can take some time to push all the data to Cloud Pub/Sub.

Understanding the push script

The code in config_geo_pubsub_push.py performs some straightforward tasks.First, the code creates a Cloud Pub/Sub client.

Next, the code finds a CSV data file and opens it.

For each line in the CSV file, the script performs a basic conversion on thelatitude and longitude values to format them in units of degrees. Then, thecode formats a timestamp based on the time information in the CSV file, andsaves the timestamp in the msg_attributes variable. After logging the valuesto the terminal window, the code formats the data into a line and publishesthe data to the Cloud Pub/Sub topic.

Pulling data from the topic

To pull the data from the topic and load it into your BigQuerytable, you run another Python script. This script uses the same setup.yamlfile that you already modified.

Running the pull script

Run the Python script that pulls data from Cloud Pub/Suband loads itinto BigQuery: python config_geo_pubsub_pull.py

You should see repeated output like this:

This output confirms that each line of data has been loaded into yourBigQuery table.

It can take some time to pull all the data from the topic. When it's done, theterminal window will stop showing lines of output as it waits for further data.You can exit the process at any time by pressing Control+C.

Understanding the pull script

When you ran config_geo_pubsub_pull.py, the script performed some importantwork, so it's worth taking a moment to review it and understand what happened.

First, the code creates a Cloud Pub/Sub client object, exactly like thepush script did. The code also sets up some configuration values, such as thesize of a message batch and some limits for geocoding operations to stay withindaily quotas.

Next, the code creates an instance of the Google Maps Platform client andcreates an HTTP POST body for Cloud Pub/Sub requests to be posted. Thefollowing code also retrieves the name of the subscription from the setup.yamlfile.

The code then enters a loop that runs until you terminate the process, such asby pressing Control+C. This loop pulls messages using the previouslycached Cloud Pub/Sub subscription name:

The code processes each message. The key point here is that the code uses theGoogle Maps Platform to reverse geocode from latitude and longitude to streetaddress. Click View on GitHub to see the rest of the data extraction andtranscoding of the data format for storage in BigQuery.

The code saves the row of data to BigQuery.

Finally, the code sends an acknowledgement for the original message and thenrepeats the loop.

Analyzing the data

Now that the you have transcoded and loaded the data intoBigQuery, you can use BigQuery to gain insights.This section of the tutorial shows you how to use the BigQueryconsole to run a few simple queries against this data.

1. Open the BigQuery Console:

2. Select the sandiego_freeways database.

3. Click the Compose Query button.

4. In the New Query text box, enter the following query that produces average speed by zip code:

You should see results like this:

Here are two more queries you can try:

Average speed by street name

Worst speeding places

Visualizing the data

You can use Google Maps to visualize the data you stored in BigQuery. Thispart of the tutorial shows you how to superimpose a heat map visualizationonto a map of theregion. The heat map shows the volume of traffic activity captured in the datain BigQuery.

To keep the tutorial straightforward, the provided example uses OAuth 2.0 toauthenticate the user for the BigQuery service. You could chooseanother approach that might be better-suited for your scenario. For example,you could export query results from BigQuery and create a staticmap layer that doesn’t require the user to authenticate againstBigQuery, or you could set up authentication by using a serviceaccount and a proxy server.

To show the data visualization, follow these steps.

Modify bqapi.html

The file named bqapi.html is the web page source file. It requires somechanges to work with your data. For these modifications, you need to use keysand credentials you created earlier. You can see these values in theGCP Console on the Credentials page.

1. Make a copy of the file named bqapi.html. You can find the file in the following directory where you installed the source code:

2. Open the file in a text editor.

3. In the following script element, in the src attribute, replace Your-Maps-API-Key with the Maps API client ID you created earlier:

4. For the clientId variable, replace Your-Client-ID with the OAuth 2.0 client ID you created earlier.

5. For the projectId variable, replace Your-Project-IDnwith your project ID.

6. Save the file.

Viewing the web page

You can serve the web page from the Python simple HTTP server. Follow thesesteps:

1. In your terminal window, navigate to the bigquery-reverse-geolocation/web directory where you cloned the source code.

2. Run the web server:

3. From your web browser, browse to the following URL.

   If your browser has a pop-up blocker, you must disable it for traffic onlocalhost:8000and then refresh the page.

4. Click Allow in the OAuth 2.0 authentication pop-up dialog. You won't have to repeat this flow in this session if, for example, you reload the web page.

5. After the map has loaded, select the rectangle tool in the upper-left corner of the map.

6. Use the tool to draw a rectangle around the entire currently visible land mass on the map.

The page shows a heat map, similar to the following map. Exactly where the heatmap regions display on the map depends on the data you loaded intoBigQuery.

Enter Control+C in your terminal window to stop serving the web page.

Understanding the web page script

The web page uses the Google Maps JavaScript API to perform its work. You sawhow the page sets up some of the configuration, such as how it references thevisualization library when you added your browser key. In this section, you takea deeper look at how the page authorizes the user, retrieves data, and rendersthe heat-map regions.

Authorizing the user

The following functions handle authentication and authorization through OAuth2.0. The function named authorise makes the request for authorization. Thefunction named handleAuthResult receives a callback from the OAuth 2.0 librarywith the result of the request. If the result is successful, the function namedloadAPI loads the BigQuery API.

Fetching the data

Recall that you draw a rectangle around the region of the map where you want tosee the heat maps. The rectangle you draw defines a set of coordinate boundariesthat restrict the subset of data to retrieve from BigQuery. Thefunction named setUpDrawingTools adds an event listener that notifies yourcode when the rectangle is drawn.

The callback is handled by the rectangleQuery function:

The rectangleQuery function calls rectangleSQL, which constructs a SQLstring based on the boundaries of the rectangle.

You can see that this function uses southwest and northeast corners of therectangle to define the boundaries for latitudes and longitudes in the dataset.For example, the longitude represented by sw.lng coincides with the left-vertical edge of the rectangle. Any longitude in the dataset that is greaterthis value would be to the right of the that edge, and therefore inside theboundaries of the rectangle. Similar logic applies to the other three sides ofthe rectangle.

The sendQuery function executes the query through the BigQuery APIby using theGoogle API Client Library for JavaScript:

The client library produces a URL and post body similar to the follow examples:

URL:

POST body:

BigQuery responds with a jobID used to poll the API to check the job statusuntil results are ready to be retrieved:

The checkJobStatus function shows how to check the status of a jobperiodically, calling thegetmethod with the jobId returned by the original query request.The function in the sample uses a 500 millisecond timeout.

The checkJobStatus function calls getQueryResults. This function uses thejobs.getQueryResultsmethod to get the results and then passes them to the doHeatMapfunction.

Showing the heat map visualization

It's important to understand that the amount of data that can be returned from aBigQuery dataset can be huge, sometimes amounting to petabytes ofdata. You must be careful to aggregate such data in a way that makes it usableso that it can be processed and displayed in a reasonable amount of time. Forexample, trying to plot the location of every row of traffic data would beuntenable in this scenario. Fortunately, the Maps API provides thevisualization.HeatmapLayer object, which is well-suited for this purpose.There are more details in the Maps APIToo Many Markers developer guide.

The doHeatMap function creates the heat map and then superimposes thevisualization onto the map that is displayed in the browser.

Additional tips

If you’re working with very large tables, your query might return too many rowsto display efficiently on a map. You can limit the results by adding a WHEREclause or a LIMIT statement to the SQL query.

BigQuery scans the entire table with every query. To optimizeyour BigQuery quota usage, select only the columns you need inyour query.

Queries run faster if you store latitude and longitude as float rather thanstring.

There are other ways to use SQL to run spatial queries against data inBigQuery, including queries that approximate a bounding circle,andUser Defined Functionsthat can be used to construct more-advanced geometry operations. There areexamples of bounding-box and circle-radius queries in theAdvanced Examples section of the BigQuery reference.

Cleaning up

After you've finished the reverse-geocoding tutorial, you can clean up the resources you created on Google Cloud Platform so you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Deleting the project

The easiest way to clean up most GCP resources is to delete theGCP Console project.

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

In the GCP Console, go to the Projects page.

Reverse Geocode Zip 4 Code

In the project list, select the project you want to delete and click Deletedelete.

Reverse Geocoding Zip 4

In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting data stored in BigQuery

To delete stored data, follow these steps:

1. Open the BigQuery Console:

2. In the left-hand panel, point to the dataset name and then click the downward-facing arrow.

3. Click Delete dataset.

4. Follow the instructions to confirm the deletion.

Deleting the Cloud Pub/Sub topic and subscription.

To delete the Cloud Pub/Sub components:

1. Open the Cloud Pub/Sub topic list page in the GCP Console:

2. In the topic list, select the checkbox for the topic.

3. Click Delete and confirm the operation.

What's next

• Try out other Google Cloud Platform features for yourself. Have a look at ourtutorials.