Learn R Programming

spartan

Coveralls: Codecov:

Simulation Parameter Analysis R Toolkit ApplicatioN: Spartan

Copy Link

Version

Install

install.packages('spartan')

Monthly Downloads

147

Version

3.0.2

License

GPL-2

Maintainer

Kieran Alden

Last Published

November 19th, 2018

Functions in spartan (3.0.2)

aa_getATestResults_overTime

Get A-Test results for multiple simulation timepoints
atest

Calculates the A-test score for two distributions
build_curve_results_from_r_object

When developing spartanDB, it became clear curve results may not be in separate files but in one R object. This takes an R object containing curve summaries and builds these into the format spartan requires to perform the analysis
check_consistency_result_type

Check that the user has declared either a file name or an R object
calculate_atest_score

Calculate the A-Test score for a parameter set in comparison with baseline
aa_summariseReplicateRuns

Summarise results in set folder structure into one single CSV file
build_performance_statistics

Internal function to build performance statistics (MSE, R2, generation time) for an emulation
check_double_value_in_range

Check that a double argument is within a specified range
check_lengths_parameters_ranges

Check that the lengths of the parameters, minimum values, and maximum values, are equal
check_column_ranges

For aleatory analysis, checks the analysis start and end columns are sensible
check_lhs_algorithm

Check that the chosen lhc sampling algorithm is either normal or optimal.
check_robustness_range_or_values

For robustness, check whether using PMIN/PMAX/PINC entry or PARAMVALS
check_confidence_interval

Check that a confidence interval is within a specified range
check_robustness_sampling_args

Pre-execution checks to perform before the spartan robustness samplng technique is executed. Checks all parameter input
check_graph_output_type

Check the requested graph types are correct (PDF, PNG, TIFF, BMP)
check_input_args

Wrapper function called by all spartan methods to check input pre-execution
aa_summariseReplicateRuns_overTime

Calculate summary responses for consistency analysis simulations at multiple timepoints
add_parameter_value_to_file

Iterates through the parameters, adding their sampled value to the netlogo experiment file
check_text

Check that an argument that should be a text label has been specified correctly
calculate_fold_MSE

Calculate the mean squared error for this fold in k-fold cross validation
check_package_installed

Check that a required package has been installed
check_text_list

Check that an arguments of a list that should be a text label has been specified correctly
aa_graphSampleSizeSummary

Plots a comparison of the maximum A-Test score for each sample size
createtest_fold

Create test data fold for k-fold cross validation
check_parameters_and_ranges

Pre-Check of the parameters and ranges specified for sampling parameter space
create_ensemble

Internal function to create the ensemble
calculate_medians_for_all_measures

Calculate medians for all measures for a simulation parameter result
dataset_precheck

Before partitioning data, removes any columns where the value is all equal, or all NA
efast_get_overall_medians_overTime

Pre-process analysis settings if multiple timepoints are being considered
aa_sampleSizeSummary

Determines the median and maximum A-Test score observed for each sample size
efast_graph_Results

Plot the parition of variance in a simulation response for each measure
check_file_exist

Checks for the existence of a file
emulate_efast_sampled_parameters

Emulate simulations for a set of eFAST generated parameter values
create_neural_network

Create neural network emulator, using neuralnet package
analysenetwork_structures

Analyse each network structure provided as a potential NN structure
generate_a_test_score

Take the first set and compare it to a distribution from another set using the A-Test
emulate_lhc_sampled_parameters

Emulate simulations for a set of latin-hypercube generated parameter values
append_time_to_argument

Appends the time to an argument if processing timepoints
check_argument_positive_int

Check that an argument that should be a positive integer has been specified correctly
check_boolean

Check that an argument that should be a boolean has been specified correctly
efast_generate_sample_netlogo

Prepares Netlogo experiment files for a variance-based sensitivity analysis, using eFAST
check_file_extension

Check the file extension of a file and return it
check_paramvals_length_equals_parameter_length

Where used in robustness analysis, check that the length of PARAMVALS equals
check_file_exists

Check whether a file exists
check_filepath_exists

Check that the filepath of required data or output exists
efast_get_overall_medians

Calculates the summary stats for each parameter set (median of any replicates)
generate_efast_parameter_sets

Use the eFAST approach to generate parameter sets
generate_parameter_table

Takes the value list and generates the sample that is output to csv file
generate_prcc_results_header

Generates the CSV file header for the prcc results file
check_robustness_parameter_and_ranges_lengths

Where used, checks that PARAMETERS, PMIN, PMAX, PINC, and BASELINE are all the same length
createTrainingFold

Create training data fold for k-fold cross validation
create_abc_settings_object

Creates ensemble-specific parameters for ABC analysis
check_list_all_integers

Check that all objects of a list are integers
efast_netlogo_get_overall_medians

Deprecated: Use efast_netlogo_get_overall_medians
getMediansSubset

Function used to calculate the median results for those run under a set of parameter samples
efast_cvmethod

Internal function used in calculating the partitions of variance in eFAST
check_nested_filepaths

Check that result filepaths under the root directory exist
efast_netlogo_run_Analysis

Deprecated: Use efast_run_Analysis
check_robustness_paramvals_contains_baseline

Checks that the parameter values specified in PARAMVALS contain the BASELINE
efast_generate_medians_for_all_parameter_subsets

Generates summary file for stochastic simulations stored in multiple files
efast_run_Analysis_overTime

Pre-process analysis settings if multiple timepoints are being considered
emulated_lhc_values

Latin-hypercube value set use to demonstrate emulated sensitivity analysis
emulation_algorithm_settings

Initialise machine-learning algorithms settings for emulation creation
efast_sd

Internal function used in calculating the partitions of variance in eFAST
generate_emulator_model

Internal function used to generate the requested emulator, and graph training performance
emulator_predictions

Used to generate predictions from an emulator, normalising data if required
emulator_parameter_evolution

Evolve parameter sets that meet a desired ensemble outcome
a_test_results

Analysed results from tutorial_consistency_set: a-test scores when sets compared
generate_emulators_and_ensemble

Generate a set of emulators and combine into an ensemble
aa_getATestResults

Calculates the A-Test scores observed for all sets, for each sample size
calculate_prcc_for_all_measures

For all measures, calculate the prcc for each parameter
calculate_prccs_all_parameters

Calculate PRCC values for all parameter-measure pairs
exemplar_sim_output

Example of a dataset output from an agent-based simulation, used in package testing
check_robustness_range_contains_baseline

Checks that the range specified by PMIN and PMAX contains the BASELINE
calculate_weights_for_ensemble_model

Internal function to calculate the weights for all emulators in the ensemble
construct_result_filename

Appends the time to an eFAST argument, if processing multiple timepoints
get_argument_correct_case

Tries upper and lower case names for input arguments
createAndEvaluateFolds

Create and evaluate folds within k-fold cross validation
extract_predictions_from_result_list

Internal function used to extract the predictions made in emulation generation from a list
initialise_netlogo_xml_file

Initialises the Netlogo setup file for this experiment
determine_optimal_neural_network_structure

Determine the optimal hidden layer structure from those provided
denormalise_dataset

Rescale normalised data back to it's original scale
generate_requested_emulations

Generate emulators for specified machine learning techniques with provided data
generate_predictions_from_emulator

Internal function to generate predictions from an emulator
generate_headers_for_atest_file

Generates headers for the A-Test summary CSV and R Object
get_correct_file_path_for_function

Gets the correct filepath for the column range input checker
generate_list_of_checks

Defines which functions to call to check an input argument.
get_medians_for_size_subsets

For a given sample size, get the median results to summarise results for all sets
get_file_and_object_argument_names

Gets the correct file and R object argument names for the input checker
check_acceptable_model_type

Internal function to check whether acceptable models have been specified
graph_Posteriors_All_Parameters

Graph posterior distributions generated for all parameters, to PDF file
join_strings_nospace

Join strings and separate with no space
join_strings_space

Join strings and separate by a space
join_strings

Join strings and separate by specified character
lhc_generatePRCoEffs_db_link

Generate Partial Rank Correlation Coefficients for parameter/response pairs for results in database
lhc_generate_lhc_sample_netlogo

Prepares Netlogo experiment files for a sampling-based sensitivity analysis, using latin-hypercube sampling
lhc_generate_netlogo_PRCoEffs

Deprecated. Use lhc_generatePRCoEffs instead
make_lhc_plot

Make the LHC output plot
lhc_generatePRCoEffs_overTime

Pre-process analysis settings if multiple timepoints are being considered
lhc_generateLHCSummary_overTime

Pre-process analysis settings if multiple timepoints are being considered
make_path

Combines a list of elements into a filepath separated by "/"
lhc_generatePRCoEffs

Generate Partial Rank Correlation Coefficients for parameter/response pairs
lhc_graphMeasuresForParameterChange_overTime

Wrapper for graphing LHC results for multiple timepoints
lhc_netlogo_graphMeasuresForParameterChange

Deprecated. Use lhc_graphMeasuresForParameterChange instead
efast_process_netlogo_result

Analyses Netlogo simulation data for parameter sets generated for eFAST
make_filename

Makes a filename from a list of strings, separated by "_"
normalise_dataset

Normalise a dataset such that all values are between 0 and 1
efast_parameterdist

Internal function used in calculating the partitions of variance in eFAST
make_graph_title

Make graph title, sub title, and file name
efast_setfreq

Internal function used in calculating the partitions of variance in eFAST
lhc_process_sample_run_subsets_overTime

Pre-process analysis settings if multiple timepoints are being considered
nsga2_set_user_params

Initialise analysis specific parameters for NSGA-2
meanSquaredError

Calculate the mean squared error between predictions and observed data
normaliseATest

Normalises the A-Test such that it is above 0.5
make_extension

Adds an extension to a filename
num.decimals

Diagnostic function used to determine number of decimal places
oat_processParamSubsets

Summarises stochastic, repeated, simulations for all robustness parameter sets into a single file.
efast_ttest

Internal function used in calculating the partitions of variance in eFAST
check_function_dependent_paramvals

Call the correct paramvals check for the calling function, as netlogo & robustness differ
oat_csv_result_file_analysis

Performs a robustness analysis for supplied simulation data, comparing simulation behaviour at different parameter values
check_global_param_sampling_args

Checks the input values for global parameter sampling techniques
check_netlogo_parameters_and_values

Checks the netlogo parameters and values are formatted correctly
check_numeric_list_values

Check that two lists are numeric, and the values of one are less than the other
format_efast_result_for_output

Joins the various results objects into an output ready format
close_and_write_netlogo_file

Close the current netlogo sample file and write out
pcor.mat

Internal function used to calculate the Partial Rank Correlation Coefficient
compare_all_values_of_parameter_to_baseline

For one parameter, compare responses for all values with those at baseline
generate_a_test_results_header

Generates the CSV file header for the A-Test results file
generate_sensitivity_indices

Generate eFAST Sensitivity Indices
efast_generate_medians_for_all_parameter_subsets_overTime

Pre-process analysis settings if multiple timepoints are being considered
oat_processParamSubsets_overTime

Summarises stochastic, repeated, simulations for all robustness parameter sets into a single file, for multiple timepoints
plot_compare_sim_observed_to_model_prediction

Internal function used to create accuracy plots of the emulation against observed data
ploteFASTSiFromTimepointFiles

Plot the Si value for all parameters for multiple simulation timepoints
read_all_curve_results

Reads results from each curve into a multi-dimensional array
generate_summary_stats_for_all_param_sets

Generate summary statistics for each value of all parameters in this analysis
perform_aTest_for_all_sim_measures

Performs A-Test to compare all simulation measures
pcor.test

Internal function used to calculate the Partial Rank Correlation Coefficient
pcor.rec

Internal function used to calculate the Partial Rank Correlation Coefficient
graph_sample_size_results

Graph the A-Test results for a sample size
import_model_result

Import a model result from either a CSV or XML file
oat_process_netlogo_result

Takes a Netlogo behaviour space file and performs a robustness analysis from that simulation data
efast_generate_sample

Generates parameter sets for variance-based eFAST Sensitivity Analysis
efast_run_Analysis

Runs the eFAST Analysis for the pre-generated summary file
produce_accuracy_plots_single_measure

Internal function used to create accuracy plots of the emulation against observed data
efast_run_Analysis_from_DB

Runs the eFAST Analysis for a set of results stored in a database
produce_atest_score_summary

Generates A-Test score summary for all sample sizes
sim_data_for_emulation

Set of parameter and response pairs for training an emulator of a simulation
ensemble_abc_wrapper

Wrapper to allow EasyABC functions to run using Ensemble
lhc_constructcoeff_dataset

Utility function used to create data structure for coefficient output
output_ggplot_graph

Output a ggplot graph in the requested formats
execute_checks

Executes the list of check functions compiled for the calling function
lhc_generateLHCSummary

Summarises simulation behaviour for each parameter set, by median of distribution of replicate runs
prepare_parameter_value_list

Prepares the parameter value list, as either an interval range or specific values can be supplied
read_from_csv

To save retyping all options, function to read CSV data
generate_ensemble_from_existing_emulations

Generate an ensemble from previously created spartan emulation objects
process_netlogo_parameter_range_info

Processes netlogo parameter information to generate names of those of interest to this analysis
selectSuitableStructure

Selects the most suitable neural network structure from the potentials made
set.nsga_sensitivity_params

Set parameters for NSGA-2 sensitivity analysis
produce_summary_for_all_values_of_parameter

For one parameter, evaluate the results of all values that parameter can take
rSquared

Calculate the R squared statistic for predictions and observed data
scale_lhc_sample

Scale the LHC design to be the range explored for each parameter
lhc_generateTimepointFiles

Generates spartan-compatible timepoint files if simulation results over time are in one file
lhc_generate_lhc_sample

Generates sets of simulation parameters using latin-hypercube sampling
screen_nsga2_parameters

Screens NSGA-2 related parameters, guiding which to select for evolving parameter sets
generate_ensemble_training_set

Internal function used to combine test set predictions from emulators to form the ensemble training set
summarise_lhc_sweep_responses

Processes an LHC sample, returning summary stats for all parameter sets
lhc_plotCoEfficients

Plots the PRCC coefficients against each other for ease of comparison
lhc_graphMeasuresForParameterChange

Generates parameter/measure plot for each pairing in the analysis
lhc_calculatePRCCForMultipleTimepoints

Calculates the PRCC for each parameter at each timepoint, storeing PRCC and P-Value in two different files to make the plot function easier
get_max_and_median_atest_scores

Return the max and median A-Test score for all measures for a sample size
kfoldCrossValidation

Perform k-fold cross validation for assessing neural network structure performance
get_median_results_for_all_measures

For a model result, calculate the medians of the desired measures
generate_model_formula

Internal function to generate the formula for training the models
generate_medians_for_param_set

Generate the median responses for a set of parameter values
lhc_graphMeasuresForParameterChange_from_db

Generates parameter/measure plot for each pairing in the analysis, from results stored in a database
lhc_process_netlogo_result

Analyses Netlogo simulations generated for a latin-hypercube based sensitivity analysis
lhc_polarplot

Creates a polar plot for each response, showing PRCC for each parameter
lhc_process_sample_run_subsets

Summarises results of runs for parameter sets generated by a latin-hypercube
oat_generate_netlogo_behaviour_space_XML

Creates a Netlogo compatible behaviour space experiment for robustness analysis
oat_graphATestsForSampleSize

Takes each parameter in turn and creates a plot showing A-Test score against parameter value.
oat_plotResultDistribution

For stochastic simulations plots the distribution of results for each parameter value
oat_parameter_sampling

Create parameter samples for robustness (local) analysis
process_parameter_value_if_exists

Process parameter value set if results exist
oat_csv_result_file_analysis_from_DB

Performs a robustness analysis for simulation results stored in a database, comparing simulation behaviour at different parameter values
oat_csv_result_file_analysis_overTime

Pre-process analysis settings if multiple timepoints are being considered
output_param_sets_per_curve

Output the generated parameter sets for each curve
partition_dataset

Partition latin-hypercube summary file to training, testing, and validation
produce_accuracy_plots_all_measures

Internal function used to create accuracy plots of the emulation against observed data, for all measures
plotATestsFromTimepointFiles

Plots the A-Tests for all timepoints being examined
plotPRCCSFromTimepointFiles

Plots Graphs for Partial Rank Correlation Coefficients Over Time
read_model_result_file

Reads a model result file, either CSV or XML
retrieve_results_for_comparison_result_set

Get the first result set, to which all others are compared
sample_parameter_space

Generate the LHC design for the chosen algorithm
summarise_replicate_runs

Summarises replicate runs of a parameter set. Used by LHC and eFAST
tutorial_consistency_set

Example dataset showing the structure for consistency analysis data
write_data_to_csv

Shortcut function for writing data to CSV file
read_simulation_results

Read in the simulation results either from a file, or R object The existance of these results was checked in pre-execution checks
updateErrorForStructure

Add the MSE for a newly examined structure to the list of those already seen
use_ensemble_to_generate_predictions

Predict simulation responses for a parameter set using an ensemble
visualise_data_distribution

Used to diagnose skew in a training dataset before use in emulation
weight_emulator_predictions_by_ensemble

Internal function to weight emulator predictions by that calculated for the ensemble
aa_graphATestsForSampleSize

Produce a plot for each sample size, showing the A-Test scores for each set of that size
aa_sampleSizeSummary_overTime

Determines median and maximum A-Test score for each sample size over time