Skip to main content

Introduction to Reproducible Data Science

Ch. 11 Working with APIs

11.1 Overview

APIs make the world go round. From airlines, to your groceries, APIs are charged with securely and efficiently shipping data back and forth between businesses and their consumers.

This example will demonstrate how to interact with a set of APIs that cover a range of possible use cases (e.g., paginated and non-paginated returns, authentication).

11.2 Be Mindful of Limits

When connecting to APIs for data, be mindful of the frequency of requests you make. With overuse, depending on the API, you may be throttled, blocked, or limited for a period of time. Check API documentation for posted limits (e.g., 1000 requests per month).

11.3 Public APIs as a Learning Tool

Below is a list of public APIs you may consider for your learning:

11.4 The Data

For this learning exercise, we will work with an API that generates random data. We will get 10 records of faux user records.

https://random-data-api.com/api/internet_stuff/random_internet_stuff

11.4.1 Custom function to call API and return data frame 

call_api <- function(api) {
return(as.data.frame(jsonlite::read_json(api)))
}
library(tidyverse)

n_records = 10
results = list()
for(i in 1:n_records) {
  results[[i]] = call_api("https://random-data-api.com/api/internet_stuff/random_internet_stuff")
}
results_df = bind_rows(results)
## 
## Attaching package: 'kableExtra'
## The following object is masked from 'package:dplyr':
## 
##     group_rows
knitr::kable(results_df)# %>% kableExtra::kable_minimal(.)
id uid email username password domain_name ip_v4_address private_ip_v4_address public_ip_v4_address ip_v4_cidr ip_v6_address ip_v6_cidr mac_address url slug user_agent
2852 5fff9e7f-22cf-432e-9c40-c9db64845ec4 jenette_renner NdD2uXg5J2 anderson-douglas.name 136.64.75.160 127.75.219.159 221.50.60.158 221.193.21.42/22 692d:b9b1:5d0:f3b1:1fa8:21be:ab8f:ffea a71:278d:84c5:e21b:86d3:fe4a:4b4:f1fd/110 9e:86:df:65:64:47 http://nader.co/mariann qui-blanditiis Mozilla/5.0 (Windows NT x.y; Win64; x64; rv:10.0) Gecko/20100101 Firefox/10.0
5775 18c58162-721e-411a-8e79-d9c3558c4be3 charita Q2xJj3TrZwF2m2S frami.co 50.90.10.126 10.93.228.221 73.193.235.83 128.104.97.156/10 bcd7:d749:5452:5b18:c0fd:1b0d:1e31:2846 7349:8625:7dbd:2d5f:1511:74d8:5263:3e94/91 4c:4c:34:57:91:bf http://kuvalis.com/coreen aut_quae Mozilla/5.0 (compatible; MSIE 9.0; AOL 9.7; AOLBuild 4343.19; Windows NT 6.1; WOW64; Trident/5.0; FunWebProducts)
5072 b9d0cde9-687e-4937-8bf0-b190fa321ae3 jame FcFkIyJs8Zp berge.info 110.170.225.18 127.244.220.153 99.2.128.162 114.37.190.177/7 43a6:d110:236b:fdf:bfbf:98e8:5753:87cd 70aa:8bc7:d5ca:c66a:9383:2fba:d1ce:9c9a/57 2c:49:ad:1b:12:c8 http://kuhlman.net/jan porro_consequatur Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16
2294 3530f9fd-2b34-4692-ba69-0ec0928ecc0b bernetta 4If5NuGdK0vV douglas.net 32.241.131.185 100.84.28.59 189.249.35.91 136.221.93.129/22 1f48:9800:c23b:d1c5:5bbd:90c1:6395:922a d6bf:99b1:a69b:c31b:c4a8:5570:fa73:ade1/80 00:14:10:09:83:b0 http://mills.name/tam nobis_aut Mozilla/5.0 (compatible; MSIE 9.0; AOL 9.7; AOLBuild 4343.19; Windows NT 6.1; WOW64; Trident/5.0; FunWebProducts)
478 abca7112-784e-4d55-a481-20cdb1e00dc8 peg_bins X8vBv8WcUnZxM0a kilback.biz 141.78.212.246 127.31.180.98 223.96.252.226 40.58.145.50/4 4efd:8c17:8715:d66e:4f7d:5a48:cdef:ab58 1e24:905:2e33:1694:7e6c:2b70:b119:8067/99 68:b4:af:68:2a:2c http://gulgowski.com/king.nienow dolorem_laboriosam Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A
780 7bd69af1-ef6b-4381-ab6e-7c9836bf383f nicolas_greenholt TkFuIzUgIj1M aufderhar.name 60.207.179.26 127.120.75.159 161.156.28.224 222.225.110.248/28 818:e893:a28b:fd50:310e:f0b8:9b7e:60b6 c9e2:6fc7:409:c34:9d91:751c:d554:451f/124 91:1a:87:48:8d:a9 http://murazik.name/eugenio qui-quasi Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
3348 263e0408-e9ef-46fc-8d20-9ac7010ae23c benton NyNkZpNcWzX jaskolski-moore.info 171.249.19.152 10.85.147.137 208.232.122.215 89.9.147.77/11 2e94:57ed:5c59:3b80:6224:729e:c9fe:5674 6ab0:64e0:c96a:b203:e843:3ea3:6077:50bd/108 d1:e9:d1:ee:a8:4f http://hilpert-wiegand.org/lynsey.cassin officia_nemo Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16
7328 96ece97e-d4a8-48bd-8f7b-550dc28d72be shalanda EzK1dGbAc0I3 gaylord.org 230.22.2.131 100.74.132.215 12.107.128.74 32.98.244.25/4 d620:8dd8:9fa8:8225:5651:58cb:b516:3e18 d64f:a511:a821:8239:d69f:65e1:a92c:34dd/71 c7:ac:0f:a7:85:53 http://cartwright.org/david voluptas-placeat Mozilla/5.0 (compatible; MSIE 9.0; AOL 9.7; AOLBuild 4343.19; Windows NT 6.1; WOW64; Trident/5.0; FunWebProducts)
5904 a269114d-4e96-4f22-bb2e-197e5a82c924 terry_yundt IkK7gXjL057s roob.co 183.11.33.85 10.75.130.155 207.24.37.41 98.108.28.46/16 ac52:b810:4744:8983:302a:aab7:c0c4:a0b b469:d323:1af1:362c:b321:ce24:a43a:c08d/108 fc:e0:50:ec:26:bc http://lynch-schmeler.org/lionel ut-repellat Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
763 2f7c085c-7b6d-4f34-aebd-7530cf78bfb0 tim Qe6CnJn433BmAlFh berge-lowe.co 57.242.5.115 127.62.30.104 1.196.17.196 221.3.211.200/20 f765:7319:f9e:f9b3:f3a1:6ad1:1f19:246a 30c4:ae09:f361:da44:b3aa:4caa:9bd0:d62c/71 5b:86:e0:7d:2b:c7 http://lind.info/hiedi.pagac dicta-et Mozilla/5.0 (compatible; MSIE 9.0; AOL 9.7; AOLBuild 4343.19; Windows NT 6.1; WOW64; Trident/5.0; FunWebProducts)