bbp_workflow.simulation.task#

Collection of simulation tasks for the Workflow engine.

Classes

`AmendSimulationCampaign`(args, *kwargs)	Generate additional simulations in the campaign with extended filter function.
`CampaignSimProcessorSbatch`(args, *kwargs)	Simulation from the campaign processor base task.
`CleanUpSimCampaignFiles`(args, *kwargs)	Clean up files from the simulation campaign folders.
`CortexNrdmsPySim`(args, *kwargs)	Cortex simulation task.
`GenerateAndRunSimCampaign`(args, *kwargs)	Generate and run simulation campaign.
`GenerateCoupledCoordsAndRunSimCampaign`(...)	Generate and run coupled coords simulation campaign.
`GenerateCoupledCoordsSimulationCampaign`(...)	Generate simulation campaign with coupled coords values.
`GenerateSimulationCampaign`(args, *kwargs)	Generate simulation campaign.
`HippocampusNrdmsPySim`(args, *kwargs)	Hippocampus simulation task.
`LookupDetailedCircuit`(args, *kwargs)	Lookup detailed circuit knowledge graph entity.
`LookupSimCampaign`(args, *kwargs)	Alias for `LookupSimulationCampaign`.
`LookupSimulationCampaign`(args, *kwargs)	Lookup simulation campaign configuration Knowledge Graph entity.
`MoveSimCampaign`(args, *kwargs)	Move simulation campaign configuration to the new path prefix.
`MultiscaleNrdmsPySim`(args, *kwargs)	Multiscale simulation task.
`NrdmsPySim`(args, *kwargs)	Base neurodamus-py simulation task.
`PlasticityCortexNrdmsPySim`(args, *kwargs)	Cortex plasticity simulation task.
`RegisterCoupledCoordsSimulationCampaign`(...)	Register an existing simulation campaign with coupled coords.
`RegisterSimCampaignFromConfig`(args, *kwargs)	Register simulation campaign configuration from the existing config.json.
`RegisterSimulationCampaign`(args, *kwargs)	Register an existing simulation campaign.
`SimCampaign`(args, *kwargs)	Run simulation campaign.
`SimCampaignHalt`(args, *kwargs)	Stop simulation campaign.
`SimCampaignInfo`(args, *kwargs)	Print simulation campaign configuration.
`SimCampaignMixin`()	Shared helper methods for old and new sim campaign tasks.
`SimCampaignThrottle`(args, *kwargs)	Change the amount of simulations running in parallel for the campaign.
`SimulationCampaign`(args, *kwargs)	Run simulation campaign.
`SimulationCampaignProcessor`(args, *kwargs)	Apply processor tasks to individual simulations form the campaign.
`ThalamusNrdmsPySim`(args, *kwargs)	Thalamus simulation task.

class bbp_workflow.simulation.task.AmendSimulationCampaign(*args, **kwargs)#

Bases: KgTask

Generate additional simulations in the campaign with extended filter function.

Usage:

simulation.cfg#

 [DEFAULT]
 module-archive: archive/2022-02

 [GenerateSimulationCampaign]
 coords: {"ca": [1.3, 1.35],
          "stim_number_mcs": [40, 60]}
 attrs: {"path_prefix": "/gpfs/bbp.cscs.ch/path/where/to/generate",
         "blue_config_template": "BlueConfig.tmpl",
         "user_target": "user.target",
         "mg": 1.0,
         "depolarization": 100.0,
         "target": "MiniColumn_0",
         "duration": 2500,
         "seed": 123456}
 meta-seed: 123
 coords_filter_func: my_module.filter_function
 param-processors: ["bbp_workflow.sci.default_depolarization_treatment",
                    "bbp_workflow.sci.default_ca_treatment"]

 [AmendSimulationCampaign]
 coords_filter_func: my_module.filter_function_ext

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation AmendSimulationCampaign \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-config

coords_filter_func#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.CampaignSimProcessorSbatch(*args, **kwargs)#

Bases: SbatchTask

Simulation from the campaign processor base task.

coords#

is_sonata#

requires()#: Require simulation.

class bbp_workflow.simulation.task.CleanUpSimCampaignFiles(*args, **kwargs)#

Bases: Task

Clean up files from the simulation campaign folders.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation CleanUpSimCampaignFiles \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id \
        files='["AllCompartments.bbp"]'

files#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.CortexNrdmsPySim(*args, **kwargs)#

Bases: NrdmsPySim

Cortex simulation task.

Run simulation defined by the simulation config located in the self.chdir folder.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation CortexNrdmsPySim \
        account=proj123 \
        chdir=/gpfs/bbp.cscs.ch/path/to/blue/config/folder

container#

modules#

class bbp_workflow.simulation.task.GenerateAndRunSimCampaign(*args, **kwargs)#

Bases: SimCampaign

Generate and run simulation campaign.

circuit_url#

requires()#: GenerateSimulationCampaign.

class bbp_workflow.simulation.task.GenerateCoupledCoordsAndRunSimCampaign(*args, **kwargs)#

Bases: GenerateAndRunSimCampaign

Generate and run coupled coords simulation campaign.

requires()#: GenerateCoupledCoordsSimulationCampaign.

class bbp_workflow.simulation.task.GenerateCoupledCoordsSimulationCampaign(*args, **kwargs)#

Bases: GenerateSimulationCampaign

Generate simulation campaign with coupled coords values.

coords#

class bbp_workflow.simulation.task.GenerateSimulationCampaign(*args, **kwargs)#

Bases: KgTask

Generate simulation campaign.

Produce simulation configuration files in unique folders at path_prefix gpfs location. Simulation configuration files are generated by replacing any blue_config_template placeholder with values from coords, attrs or any additional mapping that param-processors return. The amount of simulation configurations is determined by the Cartesian product of coords values.

As a result configuration entity will be created in the Knowledge Graph with generation activity linking to the detailed circuit.

Usage:

simulation.cfg#

 [GenerateSimulationCampaign]
 coords: {"ca": [1.3, 1.35],
          "stim_number_mcs": [40, 60]}
 attrs: {"path_prefix": "/gpfs/bbp.cscs.ch/path/where/to/generate",
         "blue_config_template": "BlueConfig.tmpl",
         "user_target": "user.target",
         "mg": 1.0,
         "depolarization": 100.0,
         "target": "MiniColumn_0",
         "duration": 2500,
         "seed": 123456}
 coords_filter_func: my_module.filter_function
 param-processors: ["bbp_workflow.sci.default_depolarization_treatment",
                    "bbp_workflow.sci.default_ca_treatment"]

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation GenerateSimulationCampaign \
        circuit-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/circuit-id

attrs#

circuit_url#

coords#

coords_filter_func#

description#

max_workers#

meta_seed#

modules#

name#

parallel_jobs#

param_processors#

post_processors#

requires()#: LookupDetailedCircuit (url=self.circuit_url).

seed_as_coord#

validate_output#

class bbp_workflow.simulation.task.HippocampusNrdmsPySim(*args, **kwargs)#

Bases: NrdmsPySim

Hippocampus simulation task.

Run simulation defined by the simulation config located in the self.chdir folder.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation HippocampusNrdmsPySim \
        account=proj123 \
        chdir=/gpfs/bbp.cscs.ch/path/to/blue/config/folder

container#

modules#

class bbp_workflow.simulation.task.LookupDetailedCircuit(*args, **kwargs)#

Bases: LookupKgEntity

Lookup detailed circuit knowledge graph entity.

output()#: KgUrlTarget( DetailedCircuit, self.url).

class bbp_workflow.simulation.task.LookupSimCampaign(*args, **kwargs)#

Bases: LookupSimulationCampaign

Alias for LookupSimulationCampaign.

class bbp_workflow.simulation.task.LookupSimulationCampaign(*args, **kwargs)#

Bases: LookupKgEntity

Lookup simulation campaign configuration Knowledge Graph entity.

output()#: KgUrlTarget( SimCampaignConf, self.url).

class bbp_workflow.simulation.task.MoveSimCampaign(*args, **kwargs)#

Bases: Task

Move simulation campaign configuration to the new path prefix.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation MoveSimCampaign \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id \
        path-prefix=/gpfs/bbp.cscs.ch/path/where/to/move

path_prefix#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.MultiscaleNrdmsPySim(*args, **kwargs)#

Bases: NrdmsPySim

Multiscale simulation task.

Run simulation defined by the simulation config located in the self.chdir folder.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation MultiscaleNrdmsPySim \
        account=proj123 \
        chdir=/gpfs/bbp.cscs.ch/path/to/blue/config/folder

command#

cpus_per_task#

modules#

ntasks_per_node#

class bbp_workflow.simulation.task.NrdmsPySim(*args, **kwargs)#

Bases: SbatchTask

Base neurodamus-py simulation task.

Run simulation defined by the BlueConfig located in the self.chdir folder. For specific circuit simulation please use ThalamusNrdmsPySim, CortexNrdmsPySim or HippocampusNrdmsPySim.

command#

container#

cpus_per_task#

enable_shm#

exclusive#

is_sonata#

mem#

model_building_steps#

output()#: RemoteTarget(path = {self.chdir}/BlueConfig.SUCCESS) or RemoteTarget(path = {self.chdir}/simulation_config.json.SUCCESS).

class bbp_workflow.simulation.task.PlasticityCortexNrdmsPySim(*args, **kwargs)#

Bases: CortexNrdmsPySim

Cortex plasticity simulation task.

Run simulation defined by the simulation config located in the self.chdir folder.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation PlasticityCortexNrdmsPySim \
        account=proj123 \
        chdir=/gpfs/bbp.cscs.ch/path/to/blue/config/folder

modules#

class bbp_workflow.simulation.task.RegisterCoupledCoordsSimulationCampaign(*args, **kwargs)#

Bases: RegisterSimulationCampaign

class bbp_workflow.simulation.task.RegisterSimCampaignFromConfig(*args, **kwargs)#

Bases: KgTask

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation RegisterSimCampaignFromConfig \
        name="Sim campaign name" \
        description="Sim campaign description" \
        kg-proj=my-nexus-project \
        config-path=/gpfs/bbp.cscs.ch/path/config.json

config_path#

description#

name#

class bbp_workflow.simulation.task.RegisterSimulationCampaign(*args, **kwargs)#

Bases: KgTask

Parameters:

folder_name (luigi.Parameter) – Folder in the path_prefix where the simulation campaign files are located.
paths (luigi.ListParameter) – Nested list of paths corresponding to the dimensions in self.coords.
coords (luigi.DictParameter) – Coordinates of the scanned values.
attrs (luigi.DictParameter) – Additional static attributes of the campaign.
name (luigi.OptionalParameter) – Name of the simulation campaign.
description (luigi.OptionalParameter) – Description of the simulation campaign.

class bbp_workflow.simulation.task.SimCampaign(*args, **kwargs)#

Bases: SimCampaignMixin, Task

Run simulation campaign.

Example below for large amount(100) of small simulations(10 processes per sim).

Usage:

simulation.cfg#

 [DEFAULT]
 kg-proj: project
 account: proj123

 [SimulationCampaign]
 exclusive: False
 mem:
 cpus-per-task: 0
 ntasks: 10
 time: 30:00
 simulation-type: CortexNrdmsPySim

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation SimCampaign \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id \
        parallel-jobs=100

benchmark#

enable_shm#

output()#: [RemoteTarget(path = cfg.SUCCESS)].

requires()#: LookupSimulationCampaign.

sim_config_url#

sim_indices#

class bbp_workflow.simulation.task.SimCampaignHalt(*args, **kwargs)#

Bases: Task

Stop simulation campaign.

Usage:

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation SimCampaignHalt \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id

force#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.SimCampaignInfo(*args, **kwargs)#

Bases: Task

Print simulation campaign configuration.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation SimCampaignInfo \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id

generate_symlinks#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.SimCampaignMixin#

Bases: object

Shared helper methods for old and new sim campaign tasks.

parallel_jobs#

class bbp_workflow.simulation.task.SimCampaignThrottle(*args, **kwargs)#

Bases: Task

Change the amount of simulations running in parallel for the campaign.

Usage:

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation SimCampaignThrottle \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id \
        parallel-jobs=20

job_id#

parallel_jobs#

requires()#: LookupSimulationCampaign.

sim_config_url#

class bbp_workflow.simulation.task.SimulationCampaign(*args, **kwargs)#

Bases: WrapperTask

Run simulation campaign.

Inherits params from SlurmCfg which are going to be propagated to the individual simulations of the campaign.

Usage:

simulation.cfg#

 [DEFAULT]
 kg-proj: project
 account: proj123

 [SimulationCampaign]
 nodes: 10
 time: 3:00:00
 model-building-steps: 2
 simulation-type: CortexNrdmsPySim

bbp-workflow launch --follow --config simulation.cfg \
    bbp_workflow.simulation SimulationCampaign \
        sim-config-url=https://bbp.epfl.ch/nexus/v1/resources/bbp/project/_/sim-cfg-id

benchmark#

cpus_per_task#

enable_shm#

exclusive#

mem#

model_building_steps#

requires()#: List of self.simulation_type.

sim_config_url#

simulation_type#

class bbp_workflow.simulation.task.SimulationCampaignProcessor(*args, **kwargs)#

Bases: WrapperTask

Apply processor tasks to individual simulations form the campaign.

coords_filter#

processor_task#

requires()#: Require simulation campaign individual processor tasks.

sim_config_url#

class bbp_workflow.simulation.task.ThalamusNrdmsPySim(*args, **kwargs)#

Bases: NrdmsPySim

Thalamus simulation task.

Run simulation defined by the simulation config located in the self.chdir folder.

Usage:

bbp-workflow launch --follow \
    bbp_workflow.simulation ThalamusNrdmsPySim \
        account=proj123 \
        chdir=/gpfs/bbp.cscs.ch/path/to/blue/config/folder

modules#