About the tibble output
Source:vignettes/articles/About-the-tibble-output.Rmd
About-the-tibble-output.RmdThe default output from an oa_fetch call is a tibble.
This object type allows each row to be one unit of entity (article,
institution, etc.), which is often helpful for downstream wrangling. It
simplifies and combines complex output elements in list columns,
which can be extracted or exploded
with dplyr::rowwise or purrr::map.
IMPORTANT: in “flattening” the list output to a
tibble, we have made a few decisions to simplify the original nested
list such as retaining only one affiliation for each author if you’re
querying Works (details in oa_df
source code). While the tibble output is sufficient in most use
cases, you may need to obtain the original nested list with all the
information on an entity for your special research problem. If so,
please specify output = "list" in your
oa_fetch call.
Example 1: institutions
Suppose we queried Open Alex to obtain information on large Canadian institutions and now want to extract their latitudes and longitudes.
institutions <- oa_fetch(
entity = "institutions",
country_code = "CA",
cited_by_count = ">4000000",
verbose = TRUE,
count_only = FALSE
)
#> Requesting url: https://api.openalex.org/institutions?filter=country_code%3ACA%2Ccited_by_count%3A%3E4000000
#> Getting 1 page of results with a total of 12 records...
head(institutions)
#> # A tibble: 6 × 22
#> id displ…¹ displ…² displ…³ inter…⁴ ror ids count…⁵ geo type
#> <chr> <chr> <list> <list> <list> <chr> <list> <chr> <lis> <chr>
#> 1 https://op… Univer… <lgl> <lgl> <df> http… <tibble> CA <df> educ…
#> 2 https://op… Univer… <lgl> <chr> <df> http… <tibble> CA <df> educ…
#> 3 https://op… McGill… <lgl> <lgl> <df> http… <tibble> CA <df> educ…
#> 4 https://op… Univer… <chr> <lgl> <df> http… <tibble> CA <df> educ…
#> 5 https://op… Univer… <lgl> <chr> <df> http… <tibble> CA <df> educ…
#> 6 https://op… Univer… <chr> <chr> <df> http… <tibble> CA <df> educ…
#> # … with 12 more variables: homepage_url <chr>, image_url <chr>,
#> # image_thumbnail_url <chr>, associated_institutions <list>,
#> # relevance_score <lgl>, works_count <int>, cited_by_count <int>,
#> # counts_by_year <list>, works_api_url <chr>, x_concepts <list>,
#> # updated_date <chr>, created_date <chr>, and abbreviated variable names
#> # ¹display_name, ²display_name_alternatives, ³display_name_acronyms,
#> # ⁴international, ⁵country_codeWe present three different approaches below.
dplyr::rowwise
The use of rowwise
used to be discouraged,
but the tidyverse team has now recognized its importance in many
workflows, and so rowwise is here to stay. We think
rowwise pairs very naturally with our list columns
output.
institutions %>%
rowwise() %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = geo$latitude,
lon = geo$longitude,
.keep = "none"
)
#> # A tibble: 12 × 4
#> # Rowwise:
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 University of Calgary I168635309 51.1 -114.
#> 6 University of Waterloo I151746483 43.5 -80.5
#> 7 Western University I125749732 43.0 -81.3
#> 8 University of Ottawa I153718931 45.4 -75.7
#> 9 University of Montreal I70931966 45.5 -73.6
#> 10 McMaster University I98251732 43.3 -79.9
#> 11 Université Laval I43406934 46.8 -71.3
#> 12 Queen's University I204722609 44.2 -76.5purrr::map
Alternatively, you can use any function in the
purrr::map family. As you can see below, the syntax is a
little less natural, but you may gain some performance
advantage if you have a really large dataframe.
institutions %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = purrr::map_dbl(geo, ~ .x$latitude),
lon = purrr::map_dbl(geo, ~ .x$longitude),
.keep = "none"
)
#> # A tibble: 12 × 4
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 University of Calgary I168635309 51.1 -114.
#> 6 University of Waterloo I151746483 43.5 -80.5
#> 7 Western University I125749732 43.0 -81.3
#> 8 University of Ottawa I153718931 45.4 -75.7
#> 9 University of Montreal I70931966 45.5 -73.6
#> 10 McMaster University I98251732 43.3 -79.9
#> 11 Université Laval I43406934 46.8 -71.3
#> 12 Queen's University I204722609 44.2 -76.5base::lapply
Similar to the purrr approach, you can use the base functions in the
lapply family, for example:
institutions %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = vapply(geo, function(x) x$latitude, numeric(1)),
lon = vapply(geo, function(x) x$longitude, numeric(1)),
.keep = "none"
)
#> # A tibble: 12 × 4
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 University of Calgary I168635309 51.1 -114.
#> 6 University of Waterloo I151746483 43.5 -80.5
#> 7 Western University I125749732 43.0 -81.3
#> 8 University of Ottawa I153718931 45.4 -75.7
#> 9 University of Montreal I70931966 45.5 -73.6
#> 10 McMaster University I98251732 43.3 -79.9
#> 11 Université Laval I43406934 46.8 -71.3
#> 12 Queen's University I204722609 44.2 -76.5Example 2: works
Suppose we have a tibble of works output and would like to find the institutions corresponding with the works’ authors. In this case, each work may have more than one affiliated institutions.
Tibble output
Assuming that each author is affiliated with only one institution, we
can call oa_fetch with the default
output = "tibble":
works <- oa_fetch(
entity = "works",
title.search = c("bibliometric analysis", "science mapping"),
cited_by_count = ">100",
from_publication_date = "2020-01-01",
to_publication_date = "2021-01-31",
sort = "cited_by_count:desc",
count_only = FALSE
)We will store the result in a list column:
multi_insts <- works %>%
rowwise() %>%
mutate(
openalex_id = stringr::str_extract(id, "W\\d+"),
institutions = list(unique(author$institution_display_name)),
.keep = "none"
)
multi_insts
#> # A tibble: 13 × 2
#> # Rowwise:
#> openalex_id institutions
#> <chr> <list>
#> 1 W3038273726 <chr [2]>
#> 2 W2990450011 <chr [2]>
#> 3 W3001491100 <chr [2]>
#> 4 W3044902155 <chr [1]>
#> 5 W3011866596 <chr [1]>
#> 6 W3000049009 <lgl [1]>
#> 7 W2990688366 <chr [3]>
#> 8 W3000910650 <chr [3]>
#> 9 W3042215340 <chr [3]>
#> 10 W2986617680 <chr [2]>
#> 11 W2999558594 <chr [2]>
#> 12 W3038187379 <chr [10]>
#> 13 W3025370095 <chr [4]>
# institutions of the first work
str(multi_insts[1, "institutions"])
#> rowws_df [1 × 1] (S3: rowwise_df/tbl_df/tbl/data.frame)
#> $ institutions:List of 1
#> ..$ : chr [1:2] "University of Southern Denmark" "BI Norwegian Business School"
#> - attr(*, "groups")= tibble [1 × 1] (S3: tbl_df/tbl/data.frame)
#> ..$ .rows: list<int> [1:1]
#> .. ..$ : int 1
#> .. ..@ ptype: int(0)List output
If we want to get all the institutions that the authors of these works are affiliated with (since one author may be affiliated with more than one institution), you would want to work with the list output.
As you can see, the nested list can be more convoluted to work with:
works_list <- oa_fetch(
entity = "works",
title.search = c("bibliometric analysis", "science mapping"),
cited_by_count = ">100",
from_publication_date = "2020-01-01",
to_publication_date = "2021-01-31",
sort = "cited_by_count:desc",
output = "list"
)
work_authors <- lapply(works_list, \(x) x$author)
unique_insts <- sapply(
work_authors,
\(z) unique(unlist(
sapply(
z, \(y) sapply(y$institutions, \(x) x$display_name)
)
))
)
unique_insts[[1]]
#> [1] "University of Southern Denmark" "BI Norwegian Business School"