" Input: {
(repo_uoa) - program repo UOA
(module_uoa) - program module UOA
(data_uoa) - program data UOA
or
(program_uoa) - useful if univeral pipeline is used, i.e. ck run pipeline:program program_uoa=...
or taken from .cm/meta.json from current directory
(random) - if 'yes', random selection of program, cmd, dataset uoa and dataset file
(to support collaborative program optimization)
(skip_local) - if 'yes', skip detection of program in a local path
(program_tags) - select programs by these tags
(program_dir) - force program directory
(target) - target machine added via 'ck add machine' with prepared target description
(useful to create farms of machines for crowd-benchmarking and crowd-tuning using CK)
(host_os) - host OS (detect, if omitted)
(target_os) - OS module to check (if omitted, analyze host)
(device_id) - device id if remote (such as adb)
(local_platform) - if 'yes', use host_os/target_os from the current platform
(useful when replaying experiments from another machine and even OS)
(prepare) - if 'yes', only prepare setup, but do not clean/compile/run program
(save_to_file) - if !='', save updated input/output (state) to this file
(skip_interaction) - if 'yes' and out=='con', skip interaction to choose parameters
(skip_device_init) - if 'yes', skip device init
(skip_info_collection) - if 'yes', skip info collection
(skip_device_info) - if 'yes', skip any device info -
useful to prepare experiment crowdsourcing packs for remote devices
Pipeline sections' settings:
(compute_platform_id) - if !='', set env['CK_COMPUTE_PLATFORM_ID']
(compute_device_id) - if !='', set env['CK_COMPUTE_DEVICE_ID']
(no_platform_features) - if 'yes', do not collect full platform features
(no_dataset_features) - if 'yes', do not search and extract data set features
(no_clean) - if 'yes', do not clean directory before compile/run
(no_compile) - if 'yes', do not compile program (useful when running the same program
under different system state conditions: CPU/GPU freq, cache/bus contentions, etc)
(compile_only_once) - if 'yes', compile only at first iteration
(no_compiler_description) - if 'yes', do not search for most close compiler description with flags ...
(no_run) - if 'yes', do not run program
useful when using autotuning to find bugs in compiler,
or find differently generated code sequencies, etc ...
(no_state_check) - do not check system/CPU state (frequency) over iterations ...
(generate_rnd_tmp_dir) - if 'yes', compile and run program in randomly generated temporal dir
or
(tmp_dir) - if !='', use this tmp_dir
(skip_clean_after) - if 'yes', do not remove run batch
(keep) - the same as skip_clean_after
(console) - if 'yes', output from program goes to console rather than file
(usually for testing/demos)
(cmd_key) - CMD key
(cmd_keys) - Select only from this list of available CMD keys
(dataset_uoa) - UOA of a dataset
(dataset_file) - dataset filename (if more than one inside one entry - suggest to have a UID in name)
(extra_dataset_tags) - list of extra data set tags (useful to set "small" during mobile phone crowdtuning)
(compiler_env_uoa) - env of a compiler
(compile_type) - static or dynamic (dynamic by default;
however takes compiler default_compile_type into account)
or
(static or dynamic)
(compiler_description_uoa) - compiler description UOA (module compiler),
if not set, there will be an attempt to detect the most close
by version
(compiler_vars) - dict with set up compiler flags (-Dvar=value) ->
they will update the ones defined as default in program description ...
(no_vars) - skip compiler vars (if you want to use default ones from the sources) ...
(remove_compiler_vars) - list of compiler vars to remove
(extra_env_for_compilation) - set environment variables before compiling program
(flags) - compile flags
(lflags) - link flags
(compiler_flags) - dict from compiler description (during autotuning),
if set, description should exist in input:choices_desc#compiler_flags# ...
(best_base_flag) - if 'yes', try to select best flag if available ...
(speed) - the same as above
(skip_best_base_flag) - if 'yes', do not use best base flag (useful for exploration of other levels -O2,-O1,etc)
(env_speed) - use environment flag for best optimization (CK_OPT_SPEED)
(shared_solution_cid) - CID-UID1-UID2 of the shared optimization solution at cKnowledge.org/repo
You can find it by clicking on a "Copy CID to clipboard" button of a given solution.
See example at http://cknowledge.org/repo/web.php?wcid=8289e0cf24346aa7:79bca2b76876b5c6
27bc42ee449e880e:79bca2b76876b5c6-8289e0cf24346aa7-f49649288ab0accd
(Ocid-uid1-uid2) Substituting compiler -Ox levels with shared solutions in above format
(-O27bc42ee449e880e:79bca2b76876b5c6-8289e0cf24346aa7-f49649288ab0accd)
(select_best_base_flag_for_first_iteration) - if 'yes' and autotuning_iteration=0
(env) - preset environment
(env.{KEY}) - set env[KEY]=value (user-friendly interface via CMD)
(deps.{KEY}) - set deps[KEY]["uoa']=value (user-friendly interface via CMD to set any given dependency)
(preset_deps) - dict with {"KEY":"UOA"} to preset dependencies
(params) - dictionary with parameters passed via pre/post processing to third-party tools
for example, to configure ARM Workload Automation
(params.{KEY}) - set params[KEY]=value (user-friendly interface via CMD)
(extra_env) - extra environment as string
(extra_run_cmd) - extra CMD (can use $#key#$ for autotuning)
(debug_run_cmd) - substitute CMD with this one - usually useful for debugging to pre-set env for all deps
(run_cmd_substitutes) - dict with substs ($#key#$=value) in run CMD (useful for CMD autotuning)
(sudo) - if 'yes', force using sudo
(otherwise, can use ${CK_SUDO_INIT}, ${CK_SUDO_PRE}, ${CK_SUDO_POST})
(affinity) - set processor affinity for tihs program run (if supported by OS - see "affinity" in OS)
examples: 0 ; 0,1 ; 0-3 ; 4-7 (the last two can be useful for ARM big.LITTLE arhictecture
(repeat) - repeat kernel via environment CT_REPEAT_MAIN if supported
(do_not_reuse_repeat) - if 'yes', do not reuse repeat across iterations - needed for dataset exploration, for example
(skip_calibration) - if 'yes', skip execution time calibration (otherwise, make it around 4.0 sec)
(calibration_time) - calibration time in string, 4.0 sec. by default
(calibration_max) - max number of iterations for calibration, 10 by default
(statistical_repetition_number) - current statistical repetition of experiment
(for example, may be used to skip compilation, if >0)
(autotuning_iteration) - (int) current autotuning iteration (automatically updated during pipeline tuning)
(the_same_dataset) - if 'yes', the dataset stays the same across iterations
so skip copying dataset to remote from 2nd iteration
(repeat_compilation) - if 'yes', force compilation, even if "statistical_repetition_number">0
(cpu_freq) - set CPU frequency, if supported (using SUDO, if also supported)
using script ck-set-cpu-online-and-frequency
if "max" - try to set to maximum using script ck-set-cpu-performance
if "min" - try to set to minimum using scrupt ck-set-cpu-powersave
(gpu_freq) - set GPU frequency, if supported (using SUDO, if also supported)
using script ck-set-gpu-online-and-frequency
if "max" - try to set to maximum using script ck-set-gpu-performance
if "min" - try to set to minimum using scrupt ck-set-gpu-powersave
(energy) - if 'yes', start energy monitoring (if supported) using script ck-set-power-sensors
(gprof) - if 'yes', use gprof to collect profile info
(perf) - if 'yes', use perf to collect hardware counters
(vtune) - if 'yes', use Intel vtune to collect hardware counters
(sim) - if 'yes', use architecture simulator (found by tags "arch","sim")
(valgrind) - if 'yes', add valgrind
(dvdt_prof) - if 'yes', run program under dividiti's OpenCL profiler
(mali_hwc) - if 'yes', attempt to extract MALI GPU hardware counters
(milepost) - if 'yes', attempt to extract static program features using Milepost GCC and cTuning CC
(milepost_out_file) - if !='', record extracted MILEPOST features to this JSON file
(compile_timeout) - (sec.) - kill compile job if too long
(run_timeout) - (sec.) - kill run job if too long
(post_process_script_uoa) - run script from this UOA
(post_process_subscript) - subscript name
(post_process_params) - (string) add params to CMD
(dependencies) - compilation dependencies
(deps_cache) - cache with resolved dependencies for reuse (if needed)
(reuse_deps) - if 'yes' reuse dependencies
(force_resolve_deps) - if 'yes', force resolve deps (useful for crowd-tuning)
(choices) - exposed choices (if any)
(choices_order) - vector of flattened choices (useful if optimizations need order
such as LLVM or using our GCC plugin iterface to reorder passes,
since 'choices' dict does not have order)
(features) - exposed features (if any)
(characteristics) - measured characteristics/features/properties (if any)
(state) - kept across pipeline iterations (for example, during autotuning/exploration)
(tmp_dir) - if temporal directory is used, return it
(useful if randomly generated, to be reused for run or further iterations)
(repeat) - kernel repeat ...
(features.platform.cpu) - CPU features/properties obtained during iterations
to check that state didn't change ...
(skip_output_validation) - skip validation of output (dangerous during auto-tuning -
some optimizations may break semantics or change accuracy)
(output_validation_repo) - output validation repo UOA (when recording new output)
(program_output_uoa) - use this UOA to check/record program output
(to have the same output entry for groups of similar programs)
(overwrite_reference_output) - if 'yes', overwrite reference output (useful if broken)
(quiet) - if 'yes', automatically provide default answer to all questions when resolving dependencies ...
(last_md5) - if !='', check if MD5 and fail if didn't change!
(last_md5_fail_text) - to recognize that pipeline failure is not really a failure,
but MD5 is the same (useful when pruning compiler flags found
during collaborative autotuning, particularly via mobile devices
(less time to prune results))
(skip_print_timers) - if 'yes', skip printing fine-grain timers after execution
(add_rnd_extension_to_bin) - if 'yes', add random extension to binary and record list
(add_save_extension_to_bin) - if 'yes', add '.save' to bin to save during cleaning ...
(install_to_env) - install dependencies to env instead of CK-TOOLS (to keep it clean)!
(safe) - safe mode when searching packages first instead of detecting already installed soft
(to have more deterministic build)
(skip_exec) - if 'yes', do not clean output files and skip exec to be able to continue
post-processing during debuging
}
Output: {
return - return code = 0, if successful
> 0, if error
(error) - error text if return > 0
!!! The same as input, but with updated information !!!
ready - if 'yes', pipeline is ready (all obligatory choices are set)
if 'no', clean/compile/run program is postponed
state - should be preserved across autotuning, active (online) learning, exploration, validation iterations
}
"