ffscrapr: Managing Cache
One of the many packages that inspired ffscrapr is Dmytro Perepolkin’s polite package, which sets out some pillars of responsible web etiquette, including introducing yourself, taking slowly, and never asking twice for the same thing.
ffscrapr gives you convenient routes to support these:
You can set a
user_agentstring with any connection to help identify yourself to the API platform (MFL actually gives you a higher rate limit if you register with them as an API Client!)
You can set
rate_limitoptions to help manage the load on the API (and to comply with any restrictions that APIs have set out)
Behind the scenes, ffscrapr is saving the results of expensive (i.e. slow, data-heavy) and stable (i.e. not likely to change rapidly) queries so that a) it can return things faster to you and b) minimize server load.
This vignette explains and helps you manage this caching.
ffscrapr’s core functions often loop over the same, stable endpoints to help return the data for each function - things like “player names/positions/teams”, “franchise IDs/names”, and “league scoring rules” are all used frequently. ffscrapr incorporates memoise package to help with this functionality, and (as of this writing) saves the results of these functions:
fleaflicker_players()- these functions are stored for 7 days
ff_league()- these functions are stored for 24 hours
ff_userleagues()- these functions are stored for one hour
dp_values()- these functions are stored for 24 hours
You can find an updated list of cached functions in the zzz.R source code page.
By default, ffscrapr will save the results of each function call (i.e. function + arguments) into memory.
If you would like to have your cache stored on your local filesystem, you can adjust this behaviour by adding the following line to your (or your project) .Rprofile, or by adding it BEFORE you library/load the package.
options(ffscrapr.cache = "filesystem")
This will instruct ffscrapr to store the cache on your machine, as dictated by the typical cache location for your operating system. You can see where this is by running the following line of code: