Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Lesson 25
Pulling It All Together: Using Smart Data

Welcome to the final Pulling It All Together project. In this lesson you will pull together all you have learned to create a robust application using Go. You will be building an API that will allow users to retrieve data from third-party APIs.

PROJECT OVERVIEW

In this lesson, you'll build an API to work with geolocation and finance information. Specifically, you and users of your application will have the following functionality:

Geolocation When you send a physical address (anywhere in the world), the API will respond with the latitude/longitude of the input address as well as other geocoding information.
Finance When you send a symbol for a stock, mutual fund, exchange-traded fund (ETF), and so forth, the API will respond with a quote of the input symbol.

At a high level, the application might seem complicated since you don't have a database for latitude/longitude relative to addresses. Moreover, you don't have a real-time database that is able to provide you with accurate financial quotes. Luckily, third-party websites offer an easy way to access this information through APIs.

For creating API functions to access geolocation data, the Google Maps API allows you to retrieve latitude/longitude data given an input address. For accessing financial data, the Yahoo Finance API allows you to retrieve a quote given an input symbol.

In other words, our API will leverage other third-party APIs to aggregate different types of data such as map data and finance data. An example where this API would be useful is for building a dashboard application where users can see current information about the stocks they follow, their current location, and various other relevant information. Other examples of possible functionalities for the API are the ability to retrieve weather data given an input address or the ability to retrieve a list of new articles given a list of interests.

For now, let's focus on the two functionalities that we established earlier (geolocation and finance). We'll continue with the design of our API.

DESIGNING OUR API

As mentioned earlier, you'll be creating two services in our API by leveraging two different third-party APIs (Yahoo and Google). Both APIs are easy to use. However, as you'll see later, the two APIs provide different ways to retrieve data from their API.

Our API must retrieve data from two other APIs, so you should look at isolating the interaction of our API with each of the two other APIs. In other words, our API should delegate retrieving both the geolocation and the finance data to another application. By doing so, our API will focus primarily on handling input requests from clients and delegating the retrieval of the information to two other applications that will respectively query the Yahoo Finance API and the Google Maps API. This will allow you to have two different applications for interacting with the Yahoo Finance API and Google Maps API, respectively. Each application is completely separate from the other. Since you are relying on third-party APIs, it is important to separate the two so that, if one application goes down, you can still retrieve data from the second API. At a high level, the architecture of our application will look like Figure 25.1.

Snapshot of architecture of our smart data app — **Figure 25.1**: Architecture of our smart data app

Clients will be able to send requests to our API for geolocation and finance data. Our API will handle the input request from the client and delegate the request to the appropriate application that will handle the interaction with the third-party API.

This is where gRPCs come into play. The App 1 and App 2 applications shown in Figure 25.1 could be implemented using gRPCs. That way, you will have a server for each application that will handle communication with the third-party API. This will also allow you to implement each server separately while running both servers under the same gRPC server.

To recap, you are looking to implement the following applications:

API—The API application will have three endpoints:
- Home: This is a simple landing endpoint for our API.
- Retrieve Geolocation: This endpoint will handle requests related to geolocation.
- Retrieve Quote: This endpoint will handle requests related to financial data.
gRPC Server—The gRPC server will handle communication with the third-party APIs.
- Server 1—Geolocation Server: This gRPC server will specifically handle the communication with the Google Maps API.
  - This server will include an RPC that will handle the communication with the third-party API.
- Server 2—Finance Server: This gRPC server will specifically handle the communication with the Yahoo Finance API.
  - This server will include an RPC that will handle the communication with the third-party API.
- Our API will be able to execute the RPCs remotely to retrieve the data and return the data to clients.

Our architecture will look like what is shown in Figure 25.2.

Snapshot of our new application architecture — **Figure 25.2**: Our new application architecture

As Figure 25.2 shows, external clients will be able to send requests to our API. Additionally, our API will then connect to the gRPC server and remotely execute the RPCs that will allow you to retrieve data from the Google Maps API and the Yahoo Finance API. As expected, the gRPC server will include our two servers, the Geolocation server and the Finance server.

IMPLEMENTING THE gRPC SERVER

Let's start by looking at how to implement the gRPC server. Although the two underlying servers will serve two different types of data and the implementation between the two is different, the design step is similar.

Roughly, you will use the following steps:

Create a basic gRPC server.
Create the Geolocation server and register it to the gRPC server.
1. Visit the Google Maps API website and gain access to the API as well as get familiar with the API.
2. Implement the necessary code in the RPC of the geolocation server to call the Google Maps API and retrieve the geolocation information.
Create the Finance server and register it to the gRPC server.
1. Visit the Yahoo Finance API website and gain access to the API as well as get familiar with the API.
2. Implement the necessary code in the RPC of the finance server to call the Yahoo Finance API and retrieve the quote information.

As discussed, you will have a single gRPC server where both the Geolocation and the Finance servers are registered. You will be able to register both servers to listen on our gRPC server for incoming requests. Leveraging the code from Lesson 24, Listing 25.1 is a first look at what our gRPC server code looks like. Save this code as main.go in the root directory for this project.

LISTING 25.1
`main.go—A first look at our gRPC server`

package main
 
import (
  "fmt"
  "google.golang.org/grpc"
  "log"
  "net"
)
 
func main() {
  fmt.Println("Smart Data Server ")
  listener, err := net.Listen("tcp", ":9997")
  if err != nil {
     log.Fatalf("failed to listen: %v", err)
  }
  grpcServer := grpc.NewServer()
  err = grpcServer.Serve(listener)
  if err != nil {
     log.Fatalf("failed to serve: %s", err)
  }
}

In this code, you create a gRPC server listening on port 9997 for incoming requests. Because the code doesn't do much yet, let's make it more useful.

We mentioned that you will implement two servers, one for geolocation and one for finance data. The Geolocation server will handle communication with the Google Maps API, and the Finance server will handle communication with the Yahoo Finance API. Each server is completely independent from the other. Let's first create the shells for both servers.

The Geolocation Server Shell

To create the Geolocation shell, let's first create the proto file shown in Listing 25.2 in the root directory for our project. As you can see in the code, you are simply defining the GeoLocationService, which includes one RPC method called getGeoLocationData. This method will be handling the requests with the Google API and communicating back to the gRPC server you created in Listing 25.1.

LISTING 25.2
`geolocation.proto`

syntax = "proto3";
package geolocation;
option go_package="./geolocation";
message Message {
  string body = 1;
}
service GeoLocationService {
  rpc getGeoLocationData(Message) returns (Message) {}
}

Next, you create a new directory under the root directory named geolocation. Run the following command in the root directory to generate the boilerplate code for the Geolocation server:

protoc --go_out=plugins=grpc:geolocation geolocation.proto

This command will generate a file, geolocation.pb.go, under geolocation/geolocation. This code includes the boilerplate code for our Geolocation server and client.

Let's register the server you just created by adding it to the main.go file you created in Listing 25.1. The update is shown in Listing 25.3.

LISTING 25.3
`Updated main.go registering the new server`

package main
 
import (
  "./geolocation/geolocation"
  "fmt"
  "google.golang.org/grpc"
  "log"
  "net"
)
 
func main() {
  fmt.Println("Smart Data Server ")
  listener, err := net.Listen("tcp", ":9997")
  if err != nil {
     log.Fatalf("failed to listen: %v", err)
  }
  grpcServer := grpc.NewServer()
  // adding geolocation server
  ch1 := geolocation.Server{}
  geolocation.RegisterGeoLocationServiceServer(grpcServer, &ch1)
 
  err = grpcServer.Serve(listener)
  if err != nil {
     log.Fatalf("failed to serve: %s", err)
  }
}

If you look closely, the main.go file didn't change except for two added instructions. In the first instruction, you create a geolocation server called ch1. In the second instruction, you register it as part of our gRPC. Keep in mind that at this point, the main.go code will throw errors if you want to run it because you need to implement the RPC method from the proto file as well as the Server struct.

In the geolocation directory where the geolocation.pb.go file is located, add the code from Listing 25.4 into a new geolocation.go file.

LISTING 25.4
`The geolocation.go file in the geolocation directory`

package geolocation
 
import (
  "golang.org/x/net/context"
  "log"
)
 
type Server struct {
}
 
func (s *Server) GetGeoLocationData(ctx context.Context, in *Message) (*Message, error) {
  log.Println("Incoming GeoLocation Request")
  return &Message{Body: "Hello"}, nil
}

Here you are simply creating an empty struct for our Server type, and you implemented a very basic implementation for the RPC GetGeoLocationData. In our example, you are returning a "Hello" message. At this point, all the errors in the main.go file should disappear.

To see everything in action, create a basic client for our gRPC. In the root directory, create a client_geolocation.go file and add the code from Listing 25.5.

LISTING 25.5
`client_geolocation.go in the root directory`

package main
 
import (
  "./geolocation/geolocation"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "log"
)
 
func main() {
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  if err != nil {
     log.Fatalf("did not connect: %s", err)
  }
  defer conn.Close()
  c := geolocation.NewGeoLocationServiceClient(conn)
  response, err := c.GetGeoLocationData(context.Background(), &geolocation.Message{Body: "Hello"})
  if err != nil {
     log.Fatalf("Error when retrieving GeoLocation Data: %s", err)
  }
  // display the response from the server
  log.Printf("Response from server: %s", response.Body)
 
}

In this code, you've leveraged the same code from the previous lesson to create a basic client. The client connects to port 9997 and then creates a new Geolocation service client (using the boilerplate code that was generated in geolocation.pb.go). Finally, you execute the RPC GetGeoLocationData on the gRPC server. Here, you are sending a message, "Hello" (and you will receive "Hello" back as well). Finally, you log the response from the gRPC server.

First, run the main.go file, then run the client_geolocation.go file. If everything goes well, you should be seeing "Hello" from the gRPC server, as shown in Figure 25.3.

Snapshot of our client and server in action — **Figure 25.3**: Our client and server in action

Accessing Google Location Data

At this point, you have implemented a very basic gRPC server that doesn't do much yet. Let's change that. As discussed, the Geolocation RPC will receive (from the client) an address, and it will query the Google Maps API to retrieve geographic information about the input address.

First, let's get access to the Google Maps API. In other words, in order to query the maps API, you need to generate an API key that will grant you access to the API and allow you to interact with it. To generate an API key, go to https://developers.google.com/maps/get-started#api-key and follow the steps to generate an API key.

Keep the API key secret and safe (we won't share ours here). The API key generated will grant access to the Google Maps API. Requests to the maps API cost money, so you need to keep your API key safe. As of this writing, signing up for a Google Cloud account came with a $200 credit.

Now that you have an API key, let's inspect how you can programmatically interact with the maps API from Go (in other words, from our RPC). Luckily, Go provides support for the Google Maps API through a package, googlemaps.github.io/maps. To see examples of how to use the package on GitHub, go to https://github.com/googlemaps/google-maps-services-go.

You can download the package using go get. Enter the following on the command line:

go get googlemaps.github.io/maps

Updating geolocation.go

Next, let's update the geolocation.go file with the code shown in Listing 25.6.

LISTING 25.6
`Updated geolocation.go file with Google Maps API call`

package geolocation
 
import (
  "encoding/json"
  "errors"
  "golang.org/x/net/context"
  "googlemaps.github.io/maps"
  "log"
)
 
type Server struct {
}
 
var GOOGLE_API_KEY string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
var STD_ERROR = errors.New("Unable to retrieve GeoLocation Data from the Google Maps API. Please check the address or try again later.")
 
func (s *Server) GetGeoLocationData(ctx context.Context, in *Message) (*Message, error) {
  log.Println("Incoming GeoLocation Request")
  c, err := maps.NewClient(maps.WithAPIKey(GOOGLE_API_KEY))
  if err != nil {
     return &Message{Body: "Error"}, STD_ERROR
  }
  r := &maps.GeocodingRequest{
     Address: in.GetBody(), // you retrieve the address from the 
                            // Message in body
  }
  geocode, err := c.Geocode(context.Background(), r)
  if err != nil {
     return &Message{Body: "Error"}, STD_ERROR
  }
  if len(geocode) < 1 { // the geocode result returned is empty due 
                        // to erroneous address
     return &Message{Body: "Error"}, STD_ERROR
  }
  geocodeJson, err := json.Marshal(geocode) // convert the geocode result to
                                            // json string so you can return it.
  if err != nil {
     return &Message{Body: "Error"}, STD_ERROR
  }
  return &Message{Body: string(geocodeJson)}, nil
}

Listing 25.6 added several things that we'll discuss step by step. First, you import the maps package, which will allow you to interact with the Google Maps API.

Next, you create two variables. The first is for the API key, where you hard-code the API key you generated from the maps API in the previous step. You'll need to replace the string assigned to GOOGLE_API_KEY with your own key. The second variable is for a standard error. One thing to keep in mind is that our RPC will mostly be receiving requests from our own API (that you will develop later) and then communicating with the Google Maps API to get the data needed for the request and sending it back to our API. To isolate errors from the Google Maps API to our own API, you will return a standard message to our API if something is not working correctly.

In the GetGeoLocationData method, you do a number of things. First, you initiate a new maps client using our API key. This client will allow you to query the maps API (given that the API key is valid). If you catch an error from the client (API key invalid or Maps API down), you return a message with the body Error and the standard error you created earlier.

Next in the GetGeoLocationData method, you create a geocoding request using the maps client. The input address is stored in the body of the input message. You simply pass that value to the GeocodingRequest function from the maps client.

In the next step, you execute the geocoding request using the Geocode function. This function will send the request to the API and retrieve the data associated with the input address and return it as a response. As usual, you check if there are any errors. If that's the case, you return an error message.

Continuing through the code, you next check for an empty geocode result (in case of an invalid address). If that's the case, then you return the standard error. If no errors are thrown, you encode the geocode result into a JSON string so that you can return to the body of the message. If any errors occur due to marshaling data, you return the standard error.

Finally, you return the valid message with the body, including the results from geocoding.

Updating client_geolocation.go

At this point, let's update client_geolocation.go to include a real address in the body of the message that you will send to the RPC. Update client_geolocation.go with the code in Listing 25.7.

LISTING 25.7
`Updating client_geolocation.go to include mapping info`

package main
 
import (
  "./geolocation/geolocation"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "log"
)
 
func main() {
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  if err != nil {
     log.Fatalf("did not connect: %s", err)
  }
  defer conn.Close()
  c := geolocation.NewGeoLocationServiceClient(conn)
  response, err := c.GetGeoLocationData(context.Background(), &geolocation.Message{Body: "123 Main Street Louisville"})
  if err != nil {
     log.Fatalf("Error when retrieving GeoLocation Data: %s", err)
  }
  // display the response from the server
  log.Printf("Response from server: %s", response.Body)
 
}

Little has changed in this code from the previous version. You replace sending a Message having a Body of "Hello" with the actual address that you want to geocode. In our example you are sending the address "123 Main Street Louisville". That way, the RPC will access the address and query the maps API and then return the geocode results in a response message.

Let's test everything out by first running the main.go file then running the client_geolocation.go file. The results of running the client are shown here. As you can see, you were able to receive the geocode results in a JSON format:

2022/02/19 18:38:01 Response from server:
[{"address_components":[{"long_name":"123","short_name":"123","types":["street_number"]},{"long_name":"West Main Street","short_name":"W Main St","types":["route"]},{"long_name":"Downtown","short_name":"Downtown","types":["neighborhood","political"]},{"long_name":"Louisville","short_name":"Louisville","types":["locality","political"]},{"long_name":"Jefferson County","short_name":"Jefferson County","types":["administrative_area_level_2","political"]},{"long_name":"Kentucky","short_name":"KY","types":["administrative_area_level_1","political"]},{"long_name":"United States","short_name":"US","types":["country","political"]},{"long_name":"40202","short_name":"40202","types":["postal_code"]},{"long_name":"1343","short_name":"1343","types":["postal_code_suffix"]}],"formatted_address":"123 W Main St, Louisville, KY 40202, USA","geometry":{"location":{"lat":38.2564611,"lng":-85.7526251},"location_type":"ROOFTOP","bounds":{"northeast":{"lat":38.2566211,"lng":-85.7525723},"southwest":{"lat":38.2563198,"lng":-85.752714}},"viewport":{"northeast":{"lat":38.2577519302915,"lng":-85.75129416970849},"southwest":{"lat":38.2550539697085,"lng":-85.75399213029151}},"types":null},"types":["premise"],"place:id":"ChIJodFBbbxyaYgRwIcceIGSRWI","partial_match":false,"plus_code":{"global_code":"","compound_code":""}}]

If you replace the input address with an invalid address (for example, replace the Louisville address with "Hello how are you"), you should see the following error message:

2022/02/19 18:43:31 Error when retrieving GeoLocation Data: rpc error: code = 
Unknown desc = Unable to retrieve GeoLocation Data from the Google Maps API. 
Please check the address or try again later.
exit status 1

At this point, you have a basic gRPC server that is able to query the Google Maps API and retrieve geocode results for an input address. You can enter additional addresses in the code to see it in action.

The Finance Server Shell

Next, you will do the same steps for the Finance server. First, in the root directory, create the finance.proto file shown in Listing 25.8. In this code you are defining the FinanceService function, which includes one rpc method called getQuoteData. This method will be handling the requests with the Yahoo Finance API and communicating back to the gRPC server you created earlier.

LISTING 25.8
`finance.proto`

syntax = "proto3";
package finance;
option go_package="./finance";
message Message {
 string body = 1;
}
service FinanceService {
  rpc getQuoteData(Message) returns (Message) {}
}

Next, create a new directory under the root directory. Name the new directory finance. With the directory in place, run the following command to generate the boilerplate code for the Finance server:

protoc --go_out=plugins=grpc:finance finance.proto

This command will generate the file finance.pb.go under finance/finance. This code includes the boilerplate code for our server.

You can now add the server you just created to the main.go file you created previously. Update main.go with the code in Listing 25.9.

LISTING 25.9
`Updated main.go with the Finance server code added`

package main
 
import (
  "./finance/finance"
  "./geolocation/geolocation"
  "fmt"
  "google.golang.org/grpc"
  "log"
  "net"
)
 
func main() {
  fmt.Println("Smart Data Server ")
  listener, err := net.Listen("tcp", ":9997")
  if err != nil {
     log.Fatalf("failed to listen: %v", err)
  }
  grpcServer := grpc.NewServer()
  // adding geolocation server
  ch1 := geolocation.Server{}
  geolocation.RegisterGeoLocationServiceServer(grpcServer, &ch1)
  // adding finance server
  ch2 := finance.Server{}
  finance.RegisterFinanceServiceServer(grpcServer, &ch2)
  err = grpcServer.Serve(listener)
  if err != nil {
     log.Fatalf("failed to serve: %s", err)
  }
}

This update to main.go added two more instructions. In the first instruction, you are creating a finance server and in the second instruction, you are registering that server in the gRPC. Notice that at this point, both the Finance server and the Geolocation server are registered in our gRPC servers.

Keep in mind that at this point, main.go will throw errors if you run it because you need to implement the rpc method from the proto file and the Server struct.

In the finance directory, add the code from Listing 25.10 into a new finance.go file.

LISTING 25.10
`finance.go`

package finance
 
import (
  "golang.org/x/net/context"
  "log"
)
 
type Server struct {
}
 
func (s *Server) GetQuoteData(ctx context.Context, in *Message) (*Message, error) {
  log.Println("Incoming Quote Request")
  return &Message{Body: "Hello"}, nil
}

This code implements the Server struct, which is empty, and provides a basic implementation for GetQuoteData. In our example, you are returning a simple "Hello" message from GetQuoteData.

At this point, all the errors in the main.go file should disappear. To see everything in action, create a basic client for our second RPC. In the root directory, create a client_finance.go file and add the code from Listing 25.11.

LISTING 25.11
`client_finance.go`

package main
 
import (
  "./finance/finance"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "log"
)
 
func main() {
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  if err != nil {
     log.Fatalf("did not connect: %s", err)
  }
  defer conn.Close()
  c := finance.NewFinanceServiceClient(conn)
  response, err := c.GetQuoteData(context.Background(), &finance.Message{Body: "Hello"})
  if err != nil {
     log.Fatalf("Error when retrieving the Quote Data: %s", err)
  }
  // display the response from the server
  log.Printf("Response from server: %s", response.Body)
}

In client_finance.go, you create a basic client. The client connects to port 9997 and then creates a new Finance service client (using the boilerplate code that was generated in finance.pb.go). Finally, you execute the RPC GetQuoteData on the gRPC server. In our example, you are sending a message "Hello" (and you will receive "Hello" back as well). Finally, you log the response from the gRPC server.

First, run the main.go file, then run the client_finance.go file. If everything goes well, you should be seeing "Hello" from the gRPC server.

Accessing Yahoo Finance Quotes

The Finance server doesn't do much yet, so let's figure out how to add the necessary code to allow GetQuoteData to query the Yahoo Finance API and retrieve the information of an input symbol. For instance, if GetQuoteData sends "AAPL" to the Yahoo Finance API, it should return all the relevant stock information about the company Apple.

In order to do that, you first need to understand how the Yahoo Finance API works. You can do that by going to www.yahoofinanceapi.com, as shown in Figure 25.4.

Snapshot of Yahoo Finance API Specification — **Figure 25.4**: Yahoo Finance API Specification

You should see a page similar to Figure 25.4, which shows how the Yahoo Finance API works and includes links to the API. Click /v6/finance/quote to expand the section, as shown in Figure 25.5.

If you see a Try it out button, as shown in Figure 25.5, then click it. This will open the endpoints so that you can change them. The quote endpoint allows you to retrieve quotes given a list of symbols, region, and language. For simplicity, you will focus on U.S. and English for region and language, but feel free to experiment with other regions and languages to see the results.

In the symbols field, remove all symbols except for AAPL, as shown in Figure 25.6. Then click Execute.

Snapshot of the expanded /v6/finance/quote option — **Figure 25.5**: The expanded /v6/finance/quote option

Snapshot of changing symbols to just AAPL — **Figure 25.6**: Changing symbols to just AAPL

As you can see, the curl (which is like a command-line version of Postman) request to make the API call is shown. You might need to scroll down to see the curl statement:

curl -X 'GET' 
  'https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols=AAPL' 
  -H 'accept: application/json'

You can also see the request URL:

https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols=AAPL

This URL is the request you made to the API and it's of type GET. If you enter this URL now, you will receive a message from the API similar to the following:

{"message":"Forbidden","hint":"Sign up for API key
www.yahoofinanceapi.com/tutorial"}

In order to use the API, you need to create a key with Yahoo. Before you do that, you can scroll down further to see an example of a response that the API will return.

Signing Up for a Yahoo Finance API Key

To use the Yahoo Finance API, you will need a key. You must sign up to create one. As of this writing, you could use your key to make up to 100 requests per day at no cost. For the purpose of this lesson, that should suffice.

To create a key, you must create an account. You can do so by clicking the Sign In button and then selecting the Create Account tab. You will be asked to enter a username (email address) and password with confirmation. Once you confirm the new account, you will be presented with a page showing your API key with a Basic plan, as shown in Figure 25.7. (The API key is blocked in this image.)

Querying the Yahoo Finance API Programmatically

The next step is to figure out how to query the Yahoo Finance API programmatically from our RPC. In the geolocation RPC, you leverage the maps package implemented in Go to query the Google Maps API. Unfortunately, you don't have such a thing for the Yahoo Finance API, so you will need to get creative.

Snapshot of the Basic subscription for the Yahoo Finance API — **Figure 25.7**: The Basic subscription for the Yahoo Finance API

As discussed earlier, you were given the curl command for the Yahoo Finance request to retrieve a quote about the Apple stock. If you go back to the page (www.yahoofinanceapi.com and expand the /v6/finance/quote API again) and execute the request to get the AAPL symbol quote, this time you will see that the curl command has been updated to include your key:

curl -X 'GET' 
  'https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols=AAPL' 
  -H 'accept: application/json' 
  -H 'X-API-KEY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

Additionally, if you change the symbol to just the AAPL symbol quote again and click Execute, this time you will see the actual response displayed by calling the API, as shown in Figure 25.8, instead of just an example.

Snapshot of the browser JSON response from the RPC call to the quote API — **Figure 25.8**: The browser JSON response from the RPC call to the quote API

The output in Figure 25.8 is what you will receive from the Yahoo Finance API. This response from the quote API endpoint is adequate to use in our program. From our RPC, you will send the symbol to the Yahoo Finance API, and you will retrieve from the browser the quote in the JSON format similar to the example in Figure 25.8.

If you have curl installed on your terminal (for macOS or Linux) or PowerShell (for Microsoft Windows), you can run the curl/HTTP command from a terminal, and you will receive the following JSON results:

User % curl -X 'GET' 
  'https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols=AAPL' 
  -H 'accept: application/json' 
  -H 'X-API-KEY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
{"quoteResponse":{"result":[{"language":"en-US","region":"US","quoteType":
"EQUITY","quoteSourceName":"Delayed Quote","triggerable":true,"currency":
"USD","exchange":"NMS","shortName":"Apple Inc.","longName":"Apple Inc.",
"messageBoardId":"finmb_24937","exchangeTimezoneName":"America/New_York",
"exchangeTimezoneShortName":"EST","gmtOffSetMilliseconds":-18000000,"market":
"us_market","esgPopulated":false,"tradeable":false,"firstTradeDateMilliseconds
":345479400000,"priceHint":2,"postMarketChangePercent":-0.47818473,
"postMarketTime":1645232400,"postMarketPrice":166.5,"postMarketChange":
-0.80000305,"regularMarketChange":-1.5800018,"regularMarketChangePercent":
-0.93557656,"regularMarketTime":1645218002,"regularMarketPrice":167.3,
"regularMarketDayHigh":170.5413,"regularMarketDayRange":"166.19 - 170.5413",
"regularMarketDayLow":166.19,"regularMarketVolume":82772674,
"regularMarketPreviousClose":168.88,"bid":166.37,"ask":166.59,"bidSize":9,
"askSize":8,"fullExchangeName":"NasdaqGS","financialCurrency":"USD",
"regularMarketOpen":169.82,"averageDailyVolume3Month":101533156,
"averageDailyVolume10Day":77665000,"fiftyTwoWeekLowChange":51.090004,
"fiftyTwoWeekLowChangePercent":0.4396352,"fiftyTwoWeekRange":"116.21
- 182.94","fiftyTwoWeekHighChange":-15.639999,"fiftyTwoWeekHighChangePercent":
-0.08549251,"fiftyTwoWeekLow":116.21,"fiftyTwoWeekHigh":182.94,"dividendDate":
1644451200,"earningsTimestamp":1643301000,"earningsTimestampStart":1651003200,
"earningsTimestampEnd":1651521600,"trailingAnnualDividendRate":0.865,
"trailingPE":27.8138,"trailingAnnualDividendYield":0.00512198,
"epsTrailingTwelveMonths":6.015,"epsForward":6.56,"epsCurrentYear":6.16,
"priceEpsCurrentYear":27.159092,"sharesOutstanding":16319399936,
"bookValue":4.402,"fiftyDayAverage":172.4622,"fiftyDayAverageChange":
-5.162201,"fiftyDayAverageChangePercent":-0.029932361,"twoHundredDayAverage":
151.1004,"twoHundredDayAverageChange":16.1996,
"twoHundredDayAverageChangePercent":0.10721084,"marketCap":2730235789312,
"forwardPE":25.50305,"priceToBook":38.005455,"sourceInterval":15,
"exchangeDataDelayedBy":0,"pageViewGrowthWeekly":0.052849803,
"averageAnalystRating":"1.8 - Buy","marketState":"CLOSED",
"displayName":"Apple","symbol":"AAPL"}],"error":null}}%
                                                      User %

You can use Go to construct an HTTP request that does the same thing as this curl command. You can leverage the net package to construct your own HTTP request and query the Yahoo Finance API. To understand what you need to do, take a look at the BuildYahooRequest function in Listing 25.12.

LISTING 25.12
`The BuildYahooRequest function`

var YAHOO_API_KEY string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
var URL = "https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols="
 
 
func BuildYahooRequest(symbol string) (*http.Response, error) {
  req, err := http.NewRequest("GET", URL+symbol, nil)
  if err != nil {
     return nil, err
  }
  req.Header.Set("Accept", "application/json")
  req.Header.Set("X-Api-Key", YAHOO_API_KEY)
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
     return nil, err
  }
  return resp, nil
 
}

Listing 25.12 introduces the BuildYahooRequest function, which takes a symbol as input. The function returns an HTTP response and an error type. Before creating the function, hard-code your Yahoo API key generated from the previous step as well as the URL of the Yahoo Finance API minus the symbol. You will want to replace the string being assigned to YAHOO_API_KEY with your own API key.

Using the http.NewRequest, you create a GET HTTP request (similar to the generated curl command) using the Yahoo Finance API web address. You are also appending the symbol to the URL when you are creating the new request. If any error is detected, you return nil and the error is generated.

Next, in the header of the new request, add the other parameters of the request. In this example, you are specifying that JSON can be accepted as output. You are also adding the API key you defined in the constant YAHOO_API_Key, which is the key you generated earlier in this lesson.

Next, you execute the HTTP request using the Do function, and you receive a response type as well as an error type. If the error returned is nil, the function returns nil as well as the error. If the error is NOT nil, then you return the response from the HTTP request you sent to the Yahoo Finance API.

This version of the BuildYahooRequest function in Listing 25.12 is very minimal, but it should be able to retrieve the quote as an input symbol.

Updating GetQuoteData

Let's go back to the finance.go file and make the necessary updates to the GetQuoteData function to start querying the Yahoo Finance API. Replace finance.go with the code in Listing 25.13.

LISTING 25.13
`Updated finance.go file with an updated GetQuoteData function`

package finance
 
import (
  "errors"
  "golang.org/x/net/context"
  "io/ioutil"
  "log"
  "net/http"
)
 
type Server struct {
}
 
var YAHOO_API_KEY string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
var URL = "https://yfapi.net/v6/finance/quote?region=US&lang=en&symbols="
var STD_ERROR = errors.New("Unable to retrieve quote from the Yahoo API. Please
check the symbol or try again later.")
 
func BuildYahooRequest(symbol string) (*http.Response, error) {
  req, err := http.NewRequest("GET", URL+symbol, nil)
  if err != nil {
     return nil, err
  }
  req.Header.Set("Accept", "application/json")
  req.Header.Set("X-Api-Key", YAHOO_API_KEY)
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
     return nil, err
  }
  return resp, nil
 
}
 
func (s *Server) GetQuoteData(ctx context.Context, in *Message)
(*Message, error) {
  log.Println("Incoming Quote Request")
  symbol := in.GetBody()
  resp, err := BuildYahooRequest(symbol)
  if err != nil {
     return &Message{Body: "Error"}, STD_ERROR
  }
  body, err := ioutil.ReadAll(resp.Body)
  if err != nil {
     return &Message{Body: "Error"}, STD_ERROR
  }
  return &Message{Body: string(body)}, nil
}

In Listing 25.13 you are doing a number of things. You add the BuildYahooRequest function to the file. You also update the GetQuoteData function with the new code that will have the RPC send a request to the Yahoo Finance API (using the BuildYahooRequest function).

In the GetQuoteData function, you first start by retrieving the symbol from the input message's body. You then provide the parsed symbol to the BuildYahooRequest function to query the Yahoo Finance API, which returns a response type as well as an error. If there is an error, you return a standard error message; otherwise, you continue within the function.

Next, you read the body from the response (resp.Body) into a variable called body that is of type []byte. If there is an error during this operation, you return a standard error message. If there is no error, you continue within the function.

Finally, you return a message with the body that you retrieved in the previous step. Notice that you are using string() to convert the values in body from a slice of bytes to a string.

Testing Our Quote RPC

At this point, you are ready to test everything. First, let's update client_finance.go to send a valid symbol (instead of a "Hello" message). Update client_finance.go with the code in Listing 25.14.

LISTING 25.14
`Updated client_finance.go file to send a stock symbol`

package main
 
import (
  "./finance/finance"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "log"
)
 
func main() {
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  if err != nil {
     log.Fatalf("did not connect: %s", err)
  }
  defer conn.Close()
  c := finance.NewFinanceServiceClient(conn)
  response, err := c.GetQuoteData(context.Background(), &finance.
Message{Body: "AAPL"})
  if err != nil {
     log.Fatalf("Error when retrieving the Quote Data: %s", err)
  }
  // display the response from the server
  log.Printf("Response from server: %s", response.Body)
}

Nothing much changed in client_finance.go except for sending the symbol for the Apple company stock ("AAPL") for the Body of the message. To see the code in action, run the main.go file, and then run client_finance.go (in a separate Terminal window). If everything is correct, you should see results similar to the following, although you won't see the same values:

2022/02/19 20:28:03 Response from server: {"quoteResponse":{"result":[{"lang
uage":"en-US","region":"US","quoteType":"EQUITY","quoteSourceName":"Delayed
Quote","triggerable":true,"currency":"USD","firstTradeDateMilliseconds":3454
79400000,"priceHint":2,"postMarketChangePercent":-0.47818473,"postMarketTime
":1645232400,"postMarketPrice":166.5,"postMarketChange":-0.80000305,"regular
MarketChange":-1.5800018,"regularMarketChangePercent":-0.93557656,"regularMa
rketTime":1645218002,"regularMarketPrice":167.3,"regularMarketDayHigh":170.5
413,"regularMarketDayRange":"166.19 -170.5413","regularMarketDayLow":166.19
,"regularMarketVolume":82772674,"regularMarketPreviousClose":168.88,"bid":16
6.37,"ask":166.59,"bidSize":9,"askSize":8,"fullExchangeName":"NasdaqGS","fin
ancialCurrency":"USD","regularMarketOpen":169.82,"averageDailyVolume3Month"
:101533156,"averageDailyVolume10Day":77665000,"fiftyTwoWeekLowChange":51.09
0004,"fiftyTwoWeekLowChangePercent":0.4396352,"fiftyTwoWeekRange":"116.21 -
182.94","fiftyTwoWeekHighChange":-15.639999,"fiftyTwoWeekHighChangePercent":-
0.08549251,"fiftyTwoWeekLow":116.21,"fiftyTwoWeekHigh":182.94,"dividendDate":1
644451200,"marketState":"CLOSED","tradeable":false,"exchange":"NMS","shortName
":"Apple Inc.","longName":"Apple Inc.","messageBoardId":"finmb_24937","exchang
eTimezoneName":"America/New_York","exchangeTimezoneShortName":"EST","gmtOffSet
Milliseconds":-18000000,"market":"us_market","esgPopulated":false,"earningsTim
estamp":1643301000,"earningsTimestampStart":1651003200,"earningsTimestampEnd":
1651521600,"trailingAnnualDividendRate":0.865,"trailingPE":27.8138,"trailingA
nnualDividendYield":0.00512198,"epsTrailingTwelveMonths":6.015,"epsForward":6
.56,"epsCurrentYear":6.16,"priceEpsCurrentYear":27.159092,"sharesOutstanding"
:16319399936,"bookValue":4.402,"fiftyDayAverage":172.4622,"fiftyDayAverageCha
nge":-5.162201,"fiftyDayAverageChangePercent":-0.029932361,"twoHundredDayAvera
ge":151.1004,"twoHundredDayAverageChange":16.1996,"twoHundredDayAverageChangeP
ercent":0.10721084,"marketCap":2730235789312,"forwardPE":25.50305,"priceToBook
":38.005455,"sourceInterval":15,"exchangeDataDelayedBy":0,"
pageViewGrowthWeekly":0.052849803,"averageAnalystRating":"1.8 -Buy",
"displayName":"Apple","symbol":"AAPL"}],"error":null}}

As you can see, the client is able to successfully execute the RPC GetQuoteData from our gRPC server.

CREATING THE API

In the previous sections, you created the gRPC servers needed for our application. To recap, you want to build an API that includes two endpoints:

The first endpoint will receive geolocation requests from external clients. The API will leverage the Geolocation server you created previously to retrieve the data. In other words, our API will be a client of our gRPC server.
The second endpoint will receive quote requests from external clients. The API will leverage the finance server you created earlier to retrieve the data. Again, our API will be a client of our gRPC.

Let's go ahead and create the API, which will expose to the outside world the functionalities you built previously. In the root directory of your project, create another file, api.go. This file will contain all the code needed for our API, as shown in Listing 25.15.

LISTING 25.15
`The initial api.go file`

package main

import (
  "fmt"
  "github.com/gorilla/mux"
  "log"
  "net/http"
)

func homePage(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome to our API for Smart Data!")
  fmt.Println("Endpoint: /")
}
 
func getGeoLocationData(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API GeoLocation Request")
 
}
func getQuote(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API Quote Request")
}
 
// handleRequests will process HTTP requests and redirect them to
// the appropriate Handle function
func handleRequests() {
  // create a router to handle our requests from the mux package.
  router := mux.NewRouter().StrictSlash(true)
  // access root page
  router.HandleFunc("/", homePage)
  router.HandleFunc("/getGeoLocationData/{address}", getGeoLocationData)
  router.HandleFunc("/getQuote/{symbol}", getQuote)
  log.Fatal(http.ListenAndServe(":11112", router))
}
func main() {
  handleRequests()
}

This is minimal code, but it does stand up our API with three endpoints. First, let's look at the handleRequests function. This function leverages the mux router you used in previous lessons to create a router for our API. Notice that you have endpoints:

/ This is a simple landing page for our API.
/getGeoLocationData This endpoint will be the one that will handle incoming geolocation requests. This endpoint requires an input address.
/getQuote This endpoint will be the one that will handle incoming finance requests. This endpoint requires an input symbol.

In the last line of the handleRequests function, the server is connected to port 11112.

Next, let's look at the homePage function. This function is very simple and displays a welcome message. The other two functions (getGeoLocationData and getQuote) are mostly empty. You are logging that there is an incoming request every time you receive one. The next step will be to implement these two functions.

Implementing the getGeoLocationData Endpoints

The last step of our project is to implement the two endpoints, getGeoLocationData and getQuote. As discussed earlier, these two endpoints will call their corresponding RPC on the gRPC server you developed previously. Specifically, getGeoLocationData will call the GeoLocation RPC from our gRPC server. In other words, our API endpoint will act as a client to the gRPC server. Thus, you can leverage the client code you built earlier (client_geolocation.go) to create the code for getGeoLocationData. The code in Listing 25.16 shows an example of the getGeoLocationData function.

LISTING 25.16
`The getGeoLocationData function`

func getGeoLocationData(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API GeoLocation Request")
  vars := mux.Vars(r)
  address := vars["address"]
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  defer conn.Close()
  if err != nil {
     fmt.Fprintln(w, "Error. Please try again later")
     return
  }
  c := geolocation.NewGeoLocationServiceClient(conn)
  response, err := c.GetGeoLocationData(context.Background(),
&geolocation.Message{Body: address})
  if err != nil {
     fmt.Fprintln(w, err)
     return
  }
  fmt.Fprintln(w, response.Body)
}

Here you have done a few things. First, you use the mux package to read the address from the incoming request, r. Second, you establish a connection with the gRPC server. Note that you defer closing the connection until the function completes. You check for errors, and if you find an error, you return a standard error in the response w and then return.

If you don't have any errors, you create a Geolocation client using the function NewGeoLocationServiceClient. You execute the RPC GetGeoLocationData. It is important that GetGeoLocationData is an RPC (it is defined as a function in Go but it is still considered an RPC; check the definition in the proto file): a remote procedure (on our gRPC server) that you are executing from the client (our API).

Notice that you are passing the address you parsed previously in the body of the input message. You check for errors from executing the RPC, and if there are any issues, you return an error message in the response w of the endpoint.

Finally, you write the results returned by the RPC call (the geolocation data from Google Maps API) in the response of your endpoint getGeoLocationData.

Implementing the getQuote Endpoint

The getQuote function will call the GetQuoteData RPC from our gRPC server. In other words, our API endpoint will act as a client to the gRPC server. Thus, you can leverage the client code you built previously (client_finance.go) to create the code for getQuote. The code in Listing 25.17 shows an example of the getQuote function.

LISTING 25.17
`The getQuote function`

func getQuote(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API Quote Request")
  vars := mux.Vars(r)
  symbol := vars["symbol"]
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  defer conn.Close()
  if err != nil {
     // issues with connecting to gRPC
     fmt.Fprintln(w, "Error. Please try again later") //provide standard 
                             // error messages instead of technical errors
     return
  }
  c := finance.NewFinanceServiceClient(conn)
  response, err := c.GetQuoteData(context.Background(), &finance.Message{Body: symbol})
  if err != nil {
     // issues with the symbol (symbol doesn't exist) or the Yahoo 
     // API is not working.
     fmt.Fprintln(w, err)
     return
  }
  fmt.Fprintln(w, response.Body) //return the quote in the body of the response
}

This code follows the same logic as the previous endpoint (getGeoLocationData). You first parse the input symbol that will be used to retrieve the quote. Next, you establish a connection with the gRPC server. In case of any issues, you return an error in the response.

If there is no error, then you continue within the function and establish a new client that will be used to execute the RPC GetQuoteData. You retrieve the response from the gRPC and pass the body of the response from the gRPC to the response w of your endpoint getQuote.

Updating the api.go File

Update the api.go file with the code from the previous two listings. The final api.go code should look like Listing 25.18.

LISTING 25.18
`The updated api.go file`

package main

import (
  "./finance/finance"
  "./geolocation/geolocation"
  "fmt"
  "github.com/gorilla/mux"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "log"
  "net/http"
)
 
func homePage(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome to our API for Smart Data!")
  fmt.Println("Endpoint: /")
}
  
func getGeoLocationData(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API GeoLocation Request")
  vars := mux.Vars(r)
  address := vars["address"]
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  defer conn.Close()
  if err != nil {
     fmt.Fprintln(w, "Error. Please try again later")
     return
  }
  c := geolocation.NewGeoLocationServiceClient(conn)
  response, err := c.GetGeoLocationData(context.Background(),
&geolocation.Message{Body: address})
  if err != nil {
     fmt.Fprintln(w, err)
     return
  }
  fmt.Fprintln(w, response.Body)
}
func getQuote(w http.ResponseWriter, r *http.Request) {
  log.Println("Incoming API Quote Request")
  vars := mux.Vars(r)
  symbol := vars["symbol"]
  var conn *grpc.ClientConn
  conn, err := grpc.Dial(":9997", grpc.WithInsecure())
  defer conn.Close()
  if err != nil {
     // issues with connecting to gRPC
     fmt.Fprintln(w, "Error. Please try again later") //provide standard 
                                 // error messages instead of technical errors
     return
  }
  c := finance.NewFinanceServiceClient(conn)
  response, err := c.GetQuoteData(context.Background(), &finance.Message{Body: symbol})
  if err != nil {
     // issues with the symbol (symbol doesn't exist) or the Yahoo API is not working.
     fmt.Fprintln(w, err)
     return
  }
  fmt.Fprintln(w, response.Body) //return the quote in the body of the response
}
 
// handleRequests will process HTTP requests and redirect them to
// the appropriate Handle function
func handleRequests() {
  // create a router to handle our requests from the mux package.
  router := mux.NewRouter().StrictSlash(true)
  // access root page
  router.HandleFunc("/", homePage)
  router.HandleFunc("/getGeoLocationData/{address}", getGeoLocationData)
  router.HandleFunc("/getQuote/{symbol}", getQuote)
  log.Fatal(http.ListenAndServe(":11112", router))
}
func main() {
  handleRequests()
}

That's it. At this point, run the main.go program first, which kicks off the gRPC server. Then, run the api.go file in a separate Terminal window, as shown in Figure 25.9.

Snapshot of running our programs in separate terminals — **Figure 25.9**: Running our programs in separate terminals

Next, you will use a browser (or Postman) to send a request to the API. First, let's try the landing endpoint, as shown in Figure 25.10. You should see this by loading your browser with the default localhost URL with the specified port from our code, localhost:11112.

Snapshot of the landing endpoint for our API — **Figure 25.10**: The landing endpoint for our API

You can also tap our getQuote endpoint with input "AAPL". You can see the JSON response in the browser when you enter the URL localhost:11112/getQuote/AAPL, as shown in Figure 25.11.

Snapshot of using the getQuote endpoint — **Figure 25.11**: Using the `getQuote` endpoint

You can change the URL to include other symbols as well. Figure 25.12 shows the result of changing the symbol to TSLA (for Tesla). The returned JSON from the URL (localhost:11112/getQuote/TSLA) will have the TSLA information.

Snapshot of getting a TSLA quote with getQuote — **Figure 25.12**: Getting a TSLA quote with `getQuote`

You can also try the other endpoint, getGeoLocationData. The results of calling getGeoLocationData using the address 19 rue jean jaures paris are shown in Figure 25.13. You can see that you used the same URL structure to call the endpoint, localhost:11112/getGeoLocationData/19 rue jean jaures paris.

Snapshot of calling the getGeoLocationData endpoint — **Figure 25.13**: Calling the `getGeoLocationData` endpoint

As one last example, Figure 25.14 also presents results for calling getGeoLocationData. This time, an address in Louisville, Kentucky, is used, and the JSON results are presented.

SUMMARY

A lot was accomplished in this lesson. You built a gRPC server that includes two servers. The first server handles geolocation data through the Google Maps API. The second server handles finance data through the Yahoo Finance API.

You built an API that exposes the services you built to the outside world. The API leverages the gRPC to get real-time data about geolocation and finance data.

If you want to build other servers that handle other types of data, the process is pretty much the same, which is to design and develop the server and the RPC that will handle the data on the gRPC server. Then, you add another endpoint to the API that will handle external requests and delegate them to the gRPC server.

You could have built the API without the gRPC server and directly retrieved the data from Yahoo and Google through our API itself. However, this process could be tedious and not scalable for extremely large projects where different teams are working on the API. In this case, having the API independent from the underlying work you do to get data from third-party APIs will allow you to divide the work between different developers/teams. Moreover, the process of how you built our two servers (Geolocation and Finance) is pretty similar (although they have different implementations), which allows you to develop a standard process for developing new servers and to expand the functionality of the API.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.