Calling the Bitstamp REST API

Bitstamp is a cryptocurrency exchange that people use to trade a cryptocurrency, such as bitcoin, for a conventional currency, such as US dollar or euro. One of the good things about Bitstamp is that it provides a REST API, which can be used to get information about the latest trades, and can also be used to send orders if you have an account.

You can find out more here: https://www.bitstamp.net/api/.

For this project, the only endpoint we are interested in is the one that gets the latest transactions that happened on the exchange. It will give us an indication of the price and of the quantity of currency exchanged in a given period of time. This endpoint can be called with the following URL: https://www.bitstamp.net/api/v2/transactions/btcusd/?time=hour.

If you paste this URL in your favorite browser, you should see a JSON array containing all of the BTC (Bitcoin)/USD (US dollar) transactions that happened during the last hour. It should look similar to this:

[
{
"date": "1534582650",
"tid": "72519377",
"price": "6488.27",
"type": "1",
"amount": "0.05000000"
},
{
"date": "1534582645",
"tid": "72519375",
"price": "6488.27",
"type": "1",
"amount": "0.01263316"
},
...
]

In the previous result, if we inspect the first transaction, we can see that 0.05 bitcoins were sold ("type": "1" means sell) at a price of 6488.27 USD for 1 BTC.

There are many Java and Scala libraries to call a REST endpoint, but to keep things simple, we are just going to use the Scala and Java SDK to call the endpoint. Start a new Scala console, and run this code:

import java.net.URL
import scala.io.Source

val transactions = Source.fromURL(new URL("https://www.bitstamp.net/api/v2/transactions/btcusd/?time=hour")).mkString

With the help of the scala.io.Source class, we can get the HTTP response in a string. This is the first building block of our program. The next thing we need to do is parse the JSON objects into a collection of Scala objects to make them easier to manipulate.

As we read the whole HTTP response in a string in one go, we need enough memory on the driver process to keep that string in the heap. You might think that it would be better to read it with InputStream, but unfortunately, it is not possible with Spark Core to split a stream of data. You would have to use Spark Streaming to do this.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset