darshan.cli package

The cli package provides a basis for building future python based command line utilities. Currently, the existing commands provide basic examples with limited functionality.

class darshan.cli.CustomHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]

Bases: HelpFormatter

darshan.cli.discover_subcommands()[source]

Enable experimental features such as aggregation methods for reports.

Parameters:

verbose (bool) – Display log of enabled features. (Default: True)

darshan.cli.main()[source]

Darshan CLI wrapper, to expose individual commands as subcommands.

Submodules

darshan.cli.file_stats module

darshan.cli.file_stats.combine_dfs(list_dfs)[source]

Combine per-job DataFrames of each Darshan log into one DataFrame.

Parameters

list_dfs : a list of DataFrames.

Returns

a single DataFrame with data from multiple Darshan logs.

darshan.cli.file_stats.first_n_recs(df, n)[source]

Filter the data to return only the first n records.

Parameters

df : a dataframe n : an int, number of rows.

Returns

a DataFrame with n rows.

darshan.cli.file_stats.get_input_logs(args)[source]
darshan.cli.file_stats.group_by_file(combined_dfs)[source]

Group data using the ‘file’ column. Additionally, calculate the total number of unique jobs accessing each file.

Parameters

combined_dfs : a DataFrame with data from multiple Darshan logs.

Returns

a DataFrame with the sum of each group.

darshan.cli.file_stats.main(args: Any | None = None)[source]

Prints file statistics on a set of input Darshan logs.

Parameters

args: command line arguments.

darshan.cli.file_stats.process_logfile(log_path, mod, filter_patterns, filter_mode)[source]

Save the statistical data from a single Darshan log file to a DataFrame.

Parameters

log_path : a string, the path to a Darshan log file. mod : a string, the Darshan module name filter_patterns: regex patterns for names to exclude/include filter_mode: whether to “exclude” or “include” the filter patterns

Returns

a single DataFrame of job statistics.

darshan.cli.file_stats.rich_print(df, mod, order_by)[source]

Pretty print the DataFrame using rich tables.

Parameters

df : a dataframe mod : a string, the Darshan module name order_by : a string, the column name of the statistical metric to sort by

darshan.cli.file_stats.setup_parser(parser: ArgumentParser)[source]

Parses the command line arguments.

Parameters

parser : command line argument parser.

darshan.cli.file_stats.sort_dfs_desc(combined_dfs, order_by)[source]

Sort data by the column name the user inputs in a descending order.

Parameters

combined_dfs : a DataFrame with data from multiple Darshan logs. order_by : a string, the column name of the statistical metric to sort by.

Returns

a DataFrame sorted in descending order by a given column.

darshan.cli.info module

The info subcommand provides a basic dump of information about the darshan log equivalent to DarshanReport.info().

darshan.cli.info.main(args=None)[source]
darshan.cli.info.setup_parser(parser=None)[source]

darshan.cli.job_stats module

darshan.cli.job_stats.combine_dfs(list_dfs)[source]

Combine per-job DataFrames of each Darshan log into one DataFrame.

Parameters

list_dfs : a list of DataFrames.

Returns

a single DataFrame with data from multiple Darshan logs.

darshan.cli.job_stats.first_n_recs(df, n)[source]

Filter the data to return only the first n records.

Parameters

df : a dataframe n : an int, number of rows.

Returns

a DataFrame with n rows.

darshan.cli.job_stats.get_input_logs(args)[source]
darshan.cli.job_stats.main(args: Any | None = None)[source]

Prints job statistics on a set of input Darshan logs.

Parameters

args: command line arguments.

darshan.cli.job_stats.process_logfile(log_path, mod, filter_patterns, filter_mode)[source]

Save the statistical data from a single Darshan log file to a DataFrame.

Parameters

log_path : a string, the path to a Darshan log file. mod : a string, the Darshan module name filter_patterns: regex patterns for names to exclude/include filter_mode: whether to “exclude” or “include” the filter patterns

Returns

a single DataFrame of job statistics.

darshan.cli.job_stats.rich_print(df, mod, order_by)[source]

Pretty print the DataFrame using rich tables.

Parameters

df : a dataframe mod : a string, the Darshan module name order_by : a string, the column name of the statistical metric to sort by

darshan.cli.job_stats.setup_parser(parser: ArgumentParser)[source]

Parses the command line arguments.

Parameters

parser : command line argument parser.

darshan.cli.job_stats.sort_dfs_desc(combined_dfs, order_by)[source]

Sort data by the column name the user inputs in a descending order.

Parameters

combined_dfs : a DataFrame with data from multiple Darshan logs. order_by : a string, the column name of the statistical metric to sort by.

Returns

a DataFrame sorted in descending order by a given column.

darshan.cli.name_records module

The name_records subcommand provides a list of all the file names list in the darshan log hash table.

darshan.cli.name_records.main(args=None)[source]
darshan.cli.name_records.setup_parser(parser=None)[source]

darshan.cli.summary module

class darshan.cli.summary.ReportData(log_path: str, enable_dxt_heatmap: bool = False, filter_patterns: List[str] | None = None, filter_mode: str = 'exclude')[source]

Bases: object

Collects all of the metadata, tables, and figures required to generate a Darshan Summary Report.

Parameters

log_path: path to a darshan log file. enable_dxt_heatmap: flag indicating whether DXT heatmaps should be enabled filter_patterns: regex patterns for names to exclude/include filter_mode: whether to “exclude” or “include” the filter patterns

build_sections()[source]

Uses figure info to generate the unique sections and places the figures in their sections.

Builds the footer string for the summary report.

static get_full_command(report: DarshanReport) → str[source]

Retrieves the full command line from the report metadata.

Parameters

report: a darshan.DarshanReport.

Returns

cmd : the full command line used to generate the darshan log.

get_header()[source]

Builds the header string for the summary report.

get_metadata_table()[source]

Builds the metadata table (in html form) for the summary report.

get_module_table()[source]

Builds the module table (in html form) for the summary report.

static get_runtime(report: DarshanReport) → str[source]

Calculates the run time from the report metadata.

Parameters

report: a darshan.DarshanReport.

Returns

runtime : the calculated executable run time.

get_stylesheet()[source]

Retrieves the locally stored CSS.

register_figures()[source]

Collects and registers all figures in the report. This is the method users can edit to alter the report contents.

Examples

To add figures to the report, there are a few basic steps:

  1. Make the function used to generate the desired figure callable within the scope of this module.

  2. Create an entry in this method that contains all of the required information for the figure. This will be described in detail below.

  3. Use the figure information to create a ReportFigure.

  4. Add the ReportFigure to ReportData.figures.

Step #1 is handled by importing the function from the module it is defined in. For step #2, each figure in the report must have the following defined:

  • Section title: the desired section for the figure to be placed. If the section title is unique to the report, a new section will be created for that figure.

  • Figure title: the title of the figure

  • Figure function: the function used to produce the figure. This must be callable within the scope of this module (step #1).

  • Figure arguments: the arguments for the figure function

Some additional details can be provided as well:

  • Figure description: description of the figure used for the caption

  • Figure width: width of the figure in pixels

To complete steps 2-4, an entry can be added to this method, where a typical entry will look like the following:

# collect all of the info in a dictionary (step #2) fig_params = {

“section_title”: “Example Section Title”, “fig_title”: “Example Title”, “fig_func”: example_module.example_function, “fig_args”: dict(report=self.report), “fig_description”: “Example Caption”, “fig_width”: 500,

} # feed the dictionary into ReportFigure (step #3) example_fig = ReportFigure(**fig_params) # add the ReportFigure to ReportData.figures (step #4) self.figures.append(example_fig)

The order of the sections and figures is based on the order in which they are placed in self.figures. Since the DXT figure(s) are added first, they show up at the very top of the figure list.

class darshan.cli.summary.ReportFigure(section_title: str, fig_title: str, fig_func: Callable | None, fig_args: dict, fig_description: str = '', fig_width: int = 500, fig_grid_area: str = '', text_only_color: str = 'red')[source]

Bases: object

Stores info for each figure in ReportData.register_figures.

Parameters

section_title : the title of the section the figure belongs to.

fig_title : the title of the figure.

fig_func : the function used to generate the figure.

fig_args : the keyword arguments used for fig_func

fig_description : description of the figure, typically used as the caption.

fig_width : the width of the figure in pixels.

fig_grid_area : figure name corresponding to grid-area definitions specified in the CSS.

generate_fig()[source]

Generate a figure using the figure data.

static get_encoded_fig(mpl_fig: Any)[source]

Encode a matplotlib figure using base64 encoding.

Parameters

mpl_fig : matplotlib.figure object.

Returns

encoded_fig : base64 encoded image.

darshan.cli.summary.main(args: Any | None = None)[source]

Generates a Darshan Summary Report.

Parameters

args: command line arguments.

darshan.cli.summary.setup_parser(parser: ArgumentParser)[source]

Configures the command line arguments.

Parameters

parser : command line argument parser.

darshan.cli.to_json module

The to_json subcommand dumps the darshan log to json format.

darshan.cli.to_json.main(args=None)[source]
darshan.cli.to_json.setup_parser(parser=None)[source]