Retrieving JSON files using APIs, ISS example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrieving JavaScript Object Notation responses using APIs\n",
    "\n",
    "Here, we'll outline a brief workflow wherein an API will be used to acquire information on when the [ISS](https://en.wikipedia.org/wiki/International_Space_Station) will next pass over Singapore. \n",
    "\n",
    "The [requests library](https://2.python-requests.org//en/v0.10.6/) will be used to retrieve information on the ISS in [JSON](http://json.org/) (JavaScript Object Notation) format. The [json library](https://docs.python.org/3/library/json.html) will then be used to work with the JSON object we retrieve. \n",
    "\n",
    "Using the `requests` library, we will use a simple `get` request to retrieve information from the OpenNotify API.\n",
    "\n",
    "[OpenNotify](http://open-notify.org/) has several API endpoints. An endpoint is a server route for retrieving specific data from an API. For example, the /comments endpoint on the reddit API might retrieve information about comments, while the /users endpoint might retrieve data about users.\n",
    "\n",
    "The endpoint we'll look at on OpenNotify is the `iss-pass.json` endpoint. This endpoint rerieves info on when the ISS will next pass above a given point (so it expects a Latitude and Longitude to be specified as input).\n",
    "\n",
    "## Introduction\n",
    "\n",
    "We'll start by importing some of the libraries we'll be using, and then using the `get` request to retrieve info from the OpenNotify API.\n",
    "\n",
    "We'll pass in [Singapore's Latitude and Longitude](https://gps-coordinates.org/singapore-latitude.php) as parameters, since this is required by the API. Afterwards, we'll check the `status_code` of our response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "import json \n",
    "\n",
    "parameters = {'lat':1.351616, \"lon\":103.808053}\n",
    "response = requests.get(\"http://api.open-notify.org/iss-pass.json\", params = parameters)\n",
    "\n",
    "status_code = response.status_code\n",
    "\n",
    "status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The request we sent out returned a response with a status code of 200, which means everything went well, and the server returned us a result (if any). [Here's](https://documentation.commvault.com/commvault/v11/article?p=45599.htm) a list of some other possible status codes.\n",
    "\n",
    "Let's print the content of the response we received."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'{\\n  \"message\": \"success\", \\n  \"request\": {\\n    \"altitude\": 100, \\n    \"datetime\": 1566373248, \\n    \"latitude\": 1.351616, \\n    \"longitude\": 103.808053, \\n    \"passes\": 5\\n  }, \\n  \"response\": [\\n    {\\n      \"duration\": 369, \\n      \"risetime\": 1566374728\\n    }, \\n    {\\n      \"duration\": 647, \\n      \"risetime\": 1566380386\\n    }, \\n    {\\n      \"duration\": 643, \\n      \"risetime\": 1566422164\\n    }, \\n    {\\n      \"duration\": 339, \\n      \"risetime\": 1566428113\\n    }, \\n    {\\n      \"duration\": 645, \\n      \"risetime\": 1566463866\\n    }\\n  ]\\n}\\n'\n"
     ]
    }
   ],
   "source": [
    "content = response.content\n",
    "\n",
    "print(content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see above that the response we received was in the form of a string object. Strings are the way we pass information back and forth through APIs, but it's hard to get the information we want out of them.\n",
    "\n",
    "## Using the JSON library\n",
    "\n",
    "The JSON format encodes data structures like lists and dictionaries as strings to ensure that machines can read them easily. JSON is the primary format for sending and receiving data through APIs.\n",
    "\n",
    "Python offers great support for JSON through its json library. We can convert lists and dictionaries to JSON, and vice versa. Our ISS Pass data, for example, is a dictionary encoded as a string in JSON format.\n",
    "\n",
    "The JSON library has two main methods:\n",
    "\n",
    "- dumps - Takes in a Python object, and converts it to a string\n",
    "- loads - Takes a JSON string, and converts it to a Python object\n",
    "\n",
    "To get the content of an API's response as a Python object, we can simply use the .json() method on the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'dict'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'message': 'success',\n",
       " 'request': {'altitude': 100,\n",
       "  'datetime': 1566373248,\n",
       "  'latitude': 1.351616,\n",
       "  'longitude': 103.808053,\n",
       "  'passes': 5},\n",
       " 'response': [{'duration': 369, 'risetime': 1566374728},\n",
       "  {'duration': 647, 'risetime': 1566380386},\n",
       "  {'duration': 643, 'risetime': 1566422164},\n",
       "  {'duration': 339, 'risetime': 1566428113},\n",
       "  {'duration': 645, 'risetime': 1566463866}]}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json_data = response.json() # Using .json() method\n",
    "\n",
    "print(type(json_data))\n",
    "json_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we got the duration and time of the next 5 passes of the ISS over Singapore, as of the time of this project.\n",
    "\n",
    "Using typical Python list/dictionary indexing, we can retrieve of the duration and time of the first and second pass.\n",
    "\n",
    "### Using datetime library to better visualize the times of next pass\n",
    "\n",
    "Note that the rise-time is in seconds since [the Epoch](https://en.wikipedia.org/wiki/Unix_time)! We will import the `datetime` library to convert this timestamp of seconds into a date and time, which will be easier for us to make sense of.\n",
    "\n",
    "We'll simply index the timestamps for the first and second pass from the `json_data` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The ISS will next pass over Singapore at 2019-08-21, on 16:05:28, as of the time of this project.\n",
      "After that, it will pass over Singapore again at 2019-08-21, on 17:39:46.\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "\n",
    "# Converting seconds time stamp into date/time:\n",
    "timestamp1 = datetime.datetime.fromtimestamp(json_data[\"response\"][0]['risetime']) \n",
    "\n",
    "date1 = timestamp1.strftime('%Y-%m-%d') # getting date from timestamp\n",
    "time1 = timestamp1.strftime('%H:%M:%S') # getting time from timestamp\n",
    "\n",
    "# Converting seconds time stamp into date/time:\n",
    "timestamp2 = datetime.datetime.fromtimestamp(json_data[\"response\"][1]['risetime'])\n",
    "\n",
    "date2 = timestamp2.strftime('%Y-%m-%d') # getting date from timestamp\n",
    "time2 = timestamp2.strftime('%H:%M:%S') # getting time from timestamp\n",
    "\n",
    "print(\"The ISS will next pass over Singapore at {}, on {}, as of the time of this project.\".format(date1, time1))\n",
    "print(\"After that, it will pass over Singapore again at {}, on {}.\".format(date2, time2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The duration of its first pass will be 369 seconds.\n",
      "The duration of its second pass will be 647 seconds.\n"
     ]
    }
   ],
   "source": [
    "first_pass_duration = json_data[\"response\"][0]['duration'] \n",
    "second_pass_duration = json_data[\"response\"][1]['duration']\n",
    "\n",
    "print(\"The duration of its first pass will be {} seconds.\".format(first_pass_duration))\n",
    "print(\"The duration of its second pass will be {} seconds.\".format(second_pass_duration))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieving Header info\n",
    "\n",
    "The server sends more than a status code and the data when it generates a response. It also sends metadata containing information on how it generated the data, and how to decode it. This information appears in the response headers. We can access it using the `.headers` property that responses have.\n",
    "\n",
    "The headers will appear as a dictionary. For now, the content-type within the headers is the most important key. It tells us the format of the response, and how to decode it. For the OpenNotify API, the format is JSON, which is why we could decode it with JSON earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Server': 'nginx/1.10.3', 'Date': 'Wed, 21 Aug 2019 07:46:23 GMT', 'Content-Type': 'application/json', 'Content-Length': '527', 'Connection': 'keep-alive', 'Via': '1.1 vegur'}\n"
     ]
    }
   ],
   "source": [
    "print(response.headers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'application/json'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "content_type = response.headers['content-type']\n",
    "\n",
    "content_type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is how to verify the format of the response you receive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing another end-point\n",
    "\n",
    "OpenNotify has one more API endpoint, astros.json. It tells us how many people are currently in space. We'll use this to retrieve info on the number of people currently in Outer Space, as well as their names!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Server': 'nginx/1.10.3', 'Date': 'Wed, 21 Aug 2019 07:46:23 GMT', 'Content-Type': 'application/json', 'Content-Length': '312', 'Connection': 'keep-alive', 'access-control-allow-origin': '*'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.get(\"http://api.open-notify.org/astros.json\") # getting info from specified endpoint\n",
    "\n",
    "response.headers # to see the 'Content-Type' key, so we can verify it's a json object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'message': 'success',\n",
       " 'people': [{'name': 'Alexey Ovchinin', 'craft': 'ISS'},\n",
       "  {'name': 'Nick Hague', 'craft': 'ISS'},\n",
       "  {'name': 'Christina Koch', 'craft': 'ISS'},\n",
       "  {'name': 'Alexander Skvortsov', 'craft': 'ISS'},\n",
       "  {'name': 'Luca Parmitano', 'craft': 'ISS'},\n",
       "  {'name': 'Andrew Morgan', 'craft': 'ISS'}],\n",
       " 'number': 6}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = response.json() # converting response to python object.\n",
    "\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are currently 6 people in space!\n"
     ]
    }
   ],
   "source": [
    "num_in_space = result['number'] # simple dictionary indexing to retrieve no. of people in space\n",
    "\n",
    "print(\"There are currently {} people in space!\".format(num_in_space))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}