Stack Overflow
Stack Overflow

Questions tagged [bigrquery]

bigquery is an R package makes it easy to work with data stored in Google BigQuery by allowing you to query BigQuery tables and retrieve metadata about your projects, datasets, tables, and jobs

The bigrquery package makes it easy to work with data stored in Google BigQuery by allowing you to query BigQuery tables and retrieve metadata about your projects, datasets, tables, and jobs

77 questions
11
votes
1 answer

Append / Insert Data Into BigQuery Table Using R's bigrquery Library

Question is straightforward. I have the following code: # authenticate bigrquery::bq_auth(path = '/Users/me/restofthepath/bigquery-credentials.json') # set my project ID and dataset name project_id <- 'mygreatprojectid' dataset_name <- 'static' #…
Canovice
  • 9,012
  • 22
  • 93
  • 211
9
votes
2 answers

Increase time before tidyverse API OAuth token expires when using bigquery?

When using bigrquery from R, the tidyverse API asks for authentication, which, once provided, allows bigquery to be accessed from the R session. Downloading results can be time consuming, and the if the oauth token expires mid transfer the download…
stevec
  • 41,291
  • 27
  • 223
  • 311
7
votes
2 answers

How to select a nested field with bigrquery using dplyr syntax?

I'd like to explore a Google Analytics 360 data with bigrquery using dplyr syntax (rather than SQL), if possible. The gist is that I want to understand user journeys—I'm interested in finding the most common sequences of pages at the user level…
Khashir
  • 341
  • 3
  • 20
6
votes
1 answer

Is there a way around casting large integers as string when querying data from BigQuery through R?

Is there a better method for dealing with large integers than casting them as strings when querying data from BigQuery through R via the API? Here's an MVE showing the problem with the integer appearing as "NA": > library(bigrquery) > > bq_str <-…
dimitriy
  • 9,077
  • 2
  • 25
  • 50
6
votes
2 answers

Query text specifies use_legacy_sql:false, while API options specify:true

I'm using standardSQL with bigrquery: library(bigrquery) project <- "" sql <- " #standardSQL SELECT ;" result <- query_exec(sql, project = project, useLegacySql = FALSE) When I run the R script I get the following…
Rory Newton
  • 61
  • 1
  • 2
5
votes
2 answers

Invalid value at 'start_index' (TYPE_UINT64), "1e+05" [invalid] issue while downloading data to R from BigQuery

I successfully connected Google BigQuery with the R environment using the bigrquery package. I have defined a sql statement which extracts a report. While using the bq_table_download function, I get the following error. Invalid value at…
marine8115
  • 588
  • 3
  • 22
5
votes
1 answer

Authentication for Bigquery using bigrquery from an R Markdown document

I am having problems using bigrquery to connect to a GCP service account from within an R Markdown document that I knit. When I attempt from the console, authentication works fine.…
mysteRious
  • 4,102
  • 2
  • 16
  • 36
5
votes
4 answers

How to load large datasets to R from BigQuery?

I have tried two ways with Bigrquery package such that library(bigrquery) library(DBI) con <- dbConnect( bigrquery::bigquery(), project = "YOUR PROJECT ID HERE", dataset = "YOUR DATASET" ) test<- dbGetQuery(con, sql, n = 10000, max_pages =…
hhh
  • 50,788
  • 62
  • 179
  • 282
4
votes
1 answer

How to resolve "insufficient authentication scopes" in R package 'bigrquery'?

Question How do I get a simple 'bigrquery' function using public data to work? I just want to get ANY 'bigrquery' function to return data from Google Cloud Platform Big Query. Background Example followed: I started by looking at this example from…
ScottyJ
  • 945
  • 11
  • 16
4
votes
1 answer

Maintain column order when uploading an R data frame to Big Query

I am uploading an R data frame to Big Query with this: bq_table_upload("project.dataset.table_name", df_name) It works, but the order of the columns is different in BQ than in R. Is it possible to get the table to inherit the order of the columns…
dimitriy
  • 9,077
  • 2
  • 25
  • 50
4
votes
1 answer

How to authenticate in Google Big Query using Bigrquery without user input using a service account

I need to run an R script using windows task scheduler, however the script includes an authentication to Big Query using Bigrquery. I have the service account authentication. When I run the script I'm asked for user input which doesn't let me run…
Andres Acosta
  • 91
  • 8
3
votes
1 answer

R bigrquery: Exceeded rate limits

I am trying to download a BigQuery data set from Google Cloud Platform into R workspace in order to analyze it, using the following code: library(bigrquery) library(DBI) library(tidyverse) library(dplyr) con = dbConnect( bigquery(), project =…
Saïd Maanan
  • 511
  • 4
  • 14
2
votes
0 answers

Error while running a large BigQuery code in R

I am running my BigQuery code in RStudio using bigrquery package and one of the lines of the query contains a large list of integers (approx. 500 numbers). The query runs perfectly on BigQuery but when I try to run the same in RStudio using the…
IchBinShreyans
  • 63
  • 7
2
votes
0 answers

Internal error when downloading results of BigQuery query via R

Collecting data from BigQuery via R is failing with an Internal Error message. For example, the following query (with specific table / project names hidden), attempts to download 2.9 million rows across 299 pages. connection <-…
Megan Stamper
  • 21
  • 2
2
votes
1 answer

Create RStudio connection that uses environment variables as defaults

I am trying to add connection snippet to the bigrquery package based on this documentation: https://db.rstudio.com/advanced/snippets/ I would like to default project name to a env var like this: library(bigrquery) con <- dbConnect( bigquery(), …
Bulat
  • 6,869
  • 1
  • 29
  • 52
1
2 3 4 5 6