pre.install: Update a source and/or installed package from a task package

Description

pre.install creates a "source package" from a "task package", ready for installation using R{} CMD INSTALL/BUILD/CHECK. patch.install can be called after a pre.install; it makes a quick modification to your already-installed version of a package, and there is then no subsequent need to re-build and re-install via RCMD. It also updates the help system with immediate effect, i.e. during the current R{} session. patch.installed is a synonym for patch.install. Consult ?"mvbutils.packaging.tools" before reading further.

Usage

# 95% of the time you just need
 # pre.install( pkg)
 # patch.install( pkg)
 # Your own hook: pre.install.hook.<>( default.list, <>, ...)
 pre.install( pkg, character.only=FALSE, force.all.docs=FALSE, Rd.version, dir.above.source=NULL, ...)
 patch.installed( pkg, character.only=FALSE, force.all.docs=FALSE, help.patch=TRUE, DLLs.only=FALSE,
     update.installed.cache=option.or.default("mvb.update.installed.cache", TRUE), pre.inst=TRUE,
     Rd.version=NULL, dir.above.source=NULL)
 patch.install(...) # actually the args are exactly the same as for 'patch.installed'

Arguments

pkg

package name. Either quoted or unquoted is OK; unquoted will be treated as quoted unless...

item

character.only
force.all.docs
help.patch
DLLs.only
default.list
...
update.installed.cache
pre.inst
Rd.version
dir.above.source

bold

Compiled code

code

pre.install( pkg=mypack, dir="holder")

Details

The minimal ingredient for pre.install is a "task package"-- a directory with a ".RData" file in it, and possibly other files. [The term "task" is used because there is an expectation that this directory will be linked into the task hierarchy maintained by cd; this might not be essential, but I haven't tested it any other way.] pre.install creates a source package in a subdirectory of task package, for first-time building via RCMD BUILD. For subsequent maintenance, you would normally just call patch.install, which calls pre.install and then updates the installed package. You can override most of the default behaviour of pre.install by providing your own hook function pre.install.hook.<> in the task-- see .OVERRIDING.DEFAULTS. [_R pre-2.10 only:_ For help conversion to work, the various R{} build tools must be in the search path, just as when you're actually building the package. Also, certain environment variables may need to be set, such as "R_LIBS". All this may not automatically be the case. If not, you should set options( rcmd.shell.setup) to a character vector of commands that will set up the path & environment variables properly, when called as part of a batch file (Windows) or shell script (Linux etc). On my (Windows) system, this is CALL SET-R-BUILD-PATH-GUTS.BAT. The changes are temporary, just while the conversion is taking place.] Compiled code{ Note: see end of this subsection for changes in R{} >= 2.12 pre/patch.install do not compile source code; you need to do that yourself. If you use RCMD to do your compilation, then you can use RCMD INSTALL; this will overwrite your installed package, and probably can't be done during an R{} session. Alternatively, you may be able to use RCMD SHLIB to create the DLL directly, which you can then copy into the "libs" subdirectory of the installed package, without needing to re-install. I haven't tried this, but colleagues have. If, like me, you pre-compile your own DLLs directly (not allowed for CRAN, but fine for distribution to other users on the same OS), then you can put the DLL into a subdirectory "inst/libs" of the task, or indeed directly into the task folder; it will still end up in the "libs" subdirectory of the installed package, even though R{} doesn't compile it. To load compiled code in your package, use library.dynam in the .onLoad function (or in the .First.lib if you really aren't going to use a namespace). The alternative dyn.load is apparently frowned on these days. After the package is built, I change my compiler settings so that the DLL is created directly in the installed package's "libs" subdirectory; this means I can use the compiler's debugger while R{} is running. To accommodate this, patch.install behaves as follows:

any new DLLs in the task package are copied to the installed package;
any DLLs in the installed package but not in the task package are deleted;
for any DLLs in both task & installed, both copies are synchronized to thenewerversion;
the source package always matches the task package (?is that correct?)

You can call patch.install( mypack, DLLs.only=TRUE) if you only want the DLL-synching step. R 2 12 changes to compiled code folders{ In R{} >= 2.12, all references above to "libs" folders in the installed package should be to "libs/<>", where "arch" is your sub-architecture. You can find that from .Platform$r_arch; for 32-bit Windows, it's "i386", etc. In the task package, compiled DLLs can be placed in (i) the task folder, (ii) an "inst/libs" folder, (iii) an "inst/libs/<>" folder; it's assumed that any DLLs not in an explicit "<>" folder are for the current architecture. All task DLLs will be synchronized as appropriate with the installed package's DLLs, which as above live in the "libs/<>" folder of the installed package; and they should all end up in an "inst/libs/<>" folder in the source package. If you are using both R{} < 2.12 and R{} >= 2.12, then bear in mind the subdir argument of pre.install, which lets you create different source packages; this is useful with precompiled DLLs, since the source package structure needs to be different. } } Package structure{ The "task directory" means the working directory when you call pre.install; cd will look after this for you. The "package directory" is the subdirectory "<>" below that, which will be created if needs be. The default behaviour of pre.install is as follows-- to change it, see Overriding defaults. A basic source package is created (no C code etc.) in a subdirectory "<>" of the current task. The package will have a DESCRIPTION file, a single R{} source file with name "<>.R" in the "R" subdirectory, possibly a "sysdata.rda" file in the same place to contain non-functions, possibly a NAMESPACE file, and a set of Rd files in the "man" subdirectory. Rd files will be auto-created from flatdoc style documentation, although precedence will be given to any pre-existing Rd files found in an "Rd" subdirectory of your task, which get copied directly into the package. Any "demo", "src", and "data" subdirectories will be copied straight to the package. An "inst" subdirectory will also be copied, but recursively (i.e. including any of its subdirectories). There is no recompilation of source code. For handling of DLLs, see below. Most objects in the task will go into the package, but there are usually a few you wouldn't want there: objects that are concerned only with how to create the package in the first place, and ephemeral system clutter such as .Random.seed. The default exceptions are: functions pre.install.hook.<>, .First.task, and .Last.task; data forced!exports, .required, .Depends, tasks, .Traceback, .packageName, last.warning, .Last.value, .Random.seed, .SavedPlots; and any character vector whose name ends with ".doc". All pre-existing files in the "man", "src", "tests", "exec", "demo", "inst", and "R" subdirectories of the source-package directory will be removed (unless you have some mlazy objects; see below). Rd files in the task's "man" directory take precedence over Rd files that are created automatically by pre.install from flatdoc-style documentation. If an .Rbuildignore file is present in the task directory, it's copied to the package directory (NB I should include a facility in the pre-install hook for this). If there is a "changes.txt" file in the task directory, it will be copied to the "inst" subdirectory of the package, as will any files in the task's own "inst" subdirectory. Similarly, any DESCRIPTION file in the task directory will be copied to the package directory, after removing any "Built:" line. If there is no DESCRIPTION file in the task directory, a default DESCRIPTION file will be created in the package directory, but you'll certainly want to edit it before CRAN release; you can also generate the DESCRIPTION file yourself via the pre.install.hook override. Any "Makefile.*" in the task directory will be copied, as will any in the "src" subdirectory (not sure why both places are allowed). No other files or subdirectories in the package directory will be created or removed, but some essential files will be modified. The package is assumed to be namespaced if any of the following apply: there is a NAMESPACE file in the task directory; there is a .onLoad function in the task; there is an "Imports" directive in the DESCRIPTION file. If a NAMESPACE file is present in the task, then it is copied directly to the package. If not but the package still looks like a namespace candidate, then pre.install will generate a NAMESPACE file by calling make.NAMESPACE, which makes reasonable guesses about what to import, export, and S3methodize. What is & isn't an S3 method is generally deduced OK (see make.NAMESPACE for gruesome details), but you can override the defaults via the pre-install hook. By default, the R{} source file will only contain functions, but you can include other objects too by naming them in the funs argument. For functions, any doc and export.me attributes are dropped, but source code is kept in the source attribute. If any of the Rd files starts with a period, e.g. ".dotty.name", it will be renamed to e.g. "01.dotty.name.Rd" to avoid some problems with RCMD. If the package is not namespaced (why not?), then any undocumented functions will receive skeletal documentation in a my.proto.package-internals.Rd file. The doco is OK for RCMD CHECK, but says little more than "don't use these functions yourself". Undocumented functions are those not found by find.documented( doctype="any")). To speed up conversion of documentation, a list of raw & converted documentation is stored in the file "doc2Rd.info.rda" in the task directory, and conversion is only done for objects whose raw documentation has changed, unless force.all.docs is TRUE. pre.install creates a file "funs.rda" in the package's "R" subdirectory, which is subsequently used by patch.installed. RCMD BUILD will omit this file (currently with a complaint, though I'm trying to fix this) but it does not cause trouble. } Package documentation{If there is a text object called "<>.package.doc", then it will be passed through doc2Rd with an extra "docType{package}" field. The first line should start "<>-package" and the corresponding ".Rd" file will be put first into the index. This is the recommended way of providing package overviews-- and, speaking as a frequently bewildered would-be user of others' packages, I urge you to make use of it! } Data objects{ Data objects are handled a bit differently to the recommendations in "R extensions" and elsewhere-- but the end result for the package user is the same, or better. The changes have been made to speed up package maintenance, and to improve useability. Specifically:

Undocumented data objects live only in the package's namespace, i.e. visible only to your functions.
Documented data objects appear both in the visible part of the package (i.e. in the search path), and in the namespace. [The R{} standard is that these should not be visible in the namespace, but this doesn't seem sensible to me.]
To document a data objectxxx, include a flat-format text objectxxx.docin your task.xxx.docitself won't appear in the packaged object, but will result in documentation forxxx.
There is no need for the user ever to calldatato access a dataset in the package, and in fact it won't work;
Big data objects can be individually lazy-loaded (see below) to save time & memory, but lazy-loading is otherwise off by default for individual data objects.

Note that the data(...) function has been pretty much obsolete since the advent of lazy-loading in R{} 2.0; see R-news #4/2. In terms of package structure, as opposed to operation, there is no "data" subdirectory. Data lives either in the "sysdata.rdb/rdx" files in the "R" subdirectory (but can still be user-visible, which is not normally the case for objects in those files), or in the "mlazy" subdirectory for those objects with individual lazy-loading. } Big data objects{Lazy-loading objects cached with mlazy are handled specially, to speed up pre.install. Such objects get their cache-files copied to "inst/mlazy", and the .onLoad is prepended with code that will load them on demand. By default, they are exported if and only if documented, and are not locked. The following objects are not packaged by default, even if mlazyed: .Random.seed, .Traceback, last.warning, and .Saved.plots. These are mlazyed automatically if options( mvb.quick.cd) is TRUE-- see cd. } Overriding defaults{ If a function pre.install.hook.<> exists in the task "<>", it will be called during pre.install. It will be passed one list-mode argument, containing default values for various installation things that can be adjusted; it should return a list with the same names. It will also be passed any ... arguments to pre.install, which can be used e.g. to set "production mode" vs "informal mode" of the end product. The hook can do two things: sort out any file issues not adequately handled by pre.install, and/or change the following elements in the list that is passed in: [object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object] There are two reasons for using a hook rather than directly setting parameters in pre.install. The first is that pre.install will calculate sensible but non-obvious default values for most things, and it is easier to change the defaults than to set them up from scratch in the call. The second is that once you have written a hook, you can forget about it-- you don't have to remember special argument values each time you call pre.install for that task. }