shared Package

autotemp Module

Autotest tempfile wrapper for mkstemp (known as tempfile here) and mkdtemp (known as tempdir).

This wrapper provides a mechanism to clean up temporary files/dirs once they are no longer need.

Files/Dirs will have a unique_id prepended to the suffix and a _autotmp_ tag appended to the prefix.

It is required that the unique_id param is supplied when a temp dir/file is created.

class autotest.client.shared.autotemp.tempdir(suffix='', unique_id=None, prefix='', dir=None)[source]

Bases: object

A wrapper for tempfile.mkdtemp

@var name: The name of the temporary dir. :return: A tempdir object example usage:

b = autotemp.tempdir(unique_id=’exemdir’) b.name # your directory b.clean() # clean up after yourself
clean()[source]

Remove the temporary dir that was created. This is also called by the destructor.

class autotest.client.shared.autotemp.tempfile(unique_id, suffix='', prefix='', dir=None, text=False)[source]

Bases: object

A wrapper for tempfile.mkstemp

Parameters:unique_id – required, a unique string to help identify what part of code created the tempfile.

@var name: The name of the temporary file. @var fd: the file descriptor of the temporary file that was created. :return: a tempfile object example usage:

t = autotemp.tempfile(unique_id=’fig’) t.name # name of file t.fd # file descriptor t.fo # file object t.clean() # clean up after yourself
clean()[source]

Remove the temporary file that was created. This is also called by the destructor.

barrier Module

base_barrier Module

exception autotest.client.shared.base_barrier.BarrierAbortError[source]

Bases: autotest.client.shared.error.BarrierError

Special BarrierError raised when an explicit abort is requested.

class autotest.client.shared.base_barrier.barrier(hostid, tag, timeout=None, port=None, listen_server=None)[source]

Bases: object

Multi-machine barrier support.

Provides multi-machine barrier mechanism. Execution stops until all members arrive at the barrier.

Implementation Details:

When a barrier is forming the master node (first in sort order) in the set accepts connections from each member of the set. As they arrive they indicate the barrier they are joining and their identifier (their hostname or IP address and optional tag). They are then asked to wait. When all members are present the master node then checks that each member is still responding via a ping/pong exchange. If this is successful then everyone has checked in at the barrier. We then tell everyone they may continue via a rlse message.

Where the master is not the first to reach the barrier the client connects will fail. Client will retry until they either succeed in connecting to master or the overall timeout is exceeded.

As an example here is the exchange for a three node barrier called ‘TAG’

MASTER CLIENT1 CLIENT2

<————-TAG C1————- ————–wait————–>

[…]

<————-TAG C2—————————– ————–wait——————————>

[…]

————–ping————–> <————-pong————— ————–ping——————————> <————-pong——————————-

—– BARRIER conditions MET —–

————–rlse————–> ————–rlse——————————>

Note that once the last client has responded to pong the barrier is implicitly deemed satisifed, they have all acknowledged their presence. If we fail to send any of the rlse messages the barrier is still a success, the failed host has effectively broken ‘right at the beginning’ of the post barrier execution window.

In addition, there is another rendezvous, that makes each slave a server and the master a client. The connection process and usage is still the same but allows barriers from machines that only have a one-way connection initiation. This is called rendezvous_servers.

For example:
if ME == SERVER:
server start

b = job.barrier(ME, ‘server-up’, 120) b.rendezvous(CLIENT, SERVER)

if ME == CLIENT:
client run

b = job.barrier(ME, ‘test-complete’, 3600) b.rendezvous(CLIENT, SERVER)

if ME == SERVER:
server stop

Any client can also request an abort of the job by setting abort=True in the rendezvous arguments.

rendezvous(*hosts, **dargs)[source]
rendezvous_servers(masterid, *hosts, **dargs)[source]
autotest.client.shared.base_barrier.get_host_from_id(hostid)[source]
class autotest.client.shared.base_barrier.listen_server(address='', port=11922)[source]

Bases: object

Manages a listening socket for barrier.

Can be used to run multiple barrier instances with the same listening socket (if they were going to listen on the same port).

Attributes:

Attr address:Address to bind to (string).
Attr port:Port to bind to.
Attr socket:Listening socket object.
close()[source]

Close the listening socket.

base_check_version Module

class autotest.client.shared.base_check_version.base_check_python_version[source]
PYTHON_BIN_GLOB_STRINGS = ['/usr/bin/python2*', '/usr/local/bin/python2*']
extract_version(path)[source]
find_desired_python()[source]

Returns the path of the desired python interpreter.

restart()[source]

base_job Module

class autotest.client.shared.base_job.TAPReport(enable, resultdir=None, global_filename='status')[source]

Bases: object

Deal with TAP reporting for the Autotest client.

job_statuses = {'ABORT': False, 'ALERT': False, 'END GOOD': True, 'ERROR': False, 'FAIL': False, 'GOOD': True, 'NOSTATUS': False, 'RUNNING': False, 'START': True, 'TEST_NA': False, 'WARN': False}
record(log_entry, indent, log_files)[source]

Append a job-level status event to self._reports_container. All events will be written to TAP log files at the end of the test run. Otherwise, it’s impossilble to determine the TAP plan.

Parameters:
  • log_entry – A string status code describing the type of status entry being recorded. It must pass log.is_valid_status to be considered valid.
  • indent – Level of the log_entry to determine the operation if log_entry.operation is not given.
  • log_files – List of full path of files the TAP report will be written to at the end of the test.
record_keyval(path, dictionary, type_tag=None)[source]

Append a key-value pairs of dictionary to self._keyval_container in TAP format. Once finished write out the keyval.tap file to the file system.

If type_tag is None, then the key must be composed of alphanumeric characters (or dashes + underscores). However, if type-tag is not null then the keys must also have “{type_tag}” as a suffix. At the moment the only valid values of type_tag are “attr” and “perf”.

Parameters:
  • path – The full path of the keyval.tap file to be created
  • dictionary – The keys and values.
  • type_tag – The type of the values
classmethod tap_ok(success, counter, message)[source]

return a TAP message string.

Parameters:
  • success – True for positive message string.
  • counter – number of TAP line in plan.
  • message – additional message to report in TAP line.
write()[source]

Write the TAP reports to files.

class autotest.client.shared.base_job.base_job(*args, **dargs)[source]

Bases: object

An abstract base class for the various autotest job classes.

Property autodir:
 The top level autotest directory.
Property clientdir:
 The autotest client directory.
Property serverdir:
 The autotest server directory. [OPTIONAL]
Property resultdir:
 The directory where results should be written out. [WRITABLE]
Property pkgdir:
 The job packages directory. [WRITABLE]
Property tmpdir:
 The job temporary directory. [WRITABLE]
Property testdir:
 The job test directory. [WRITABLE]
Property customtestdir:
 The custom test directory. [WRITABLE]
Property site_testdir:
 The job site test directory. [WRITABLE]
Property bindir:
 The client bin/ directory.
Property configdir:
 The client config/ directory.
Property profdir:
 The client profilers/ directory.
Property toolsdir:
 The client tools/ directory.
Property conmuxdir:
 The conmux directory. [OPTIONAL]
Property control:
 A path to the control file to be executed. [OPTIONAL]
Property hosts:A set of all live Host objects currently in use by the job. Code running in the context of a local client can safely assume that this set contains only a single entry.
Property machines:
 A list of the machine names associated with the job.
Property user:The user executing the job.
Property tag:A tag identifying the job. Often used by the scheduler to give a name of the form NUMBER-USERNAME/HOSTNAME.
Property args:A list of additional miscellaneous command-line arguments provided when starting the job.
Property last_boot_tag:
 The label of the kernel from the last reboot. [OPTIONAL,PERSISTENT]
Property automatic_test_tag:
 A string which, if set, will be automatically added to the test name when running tests.
Property default_profile_only:
 A boolean indicating the default value of profile_only used by test.execute. [PERSISTENT]
Property drop_caches:
 A boolean indicating if caches should be dropped before each test is executed.
Property drop_caches_between_iterations:
 A boolean indicating if caches should be dropped before each test iteration is executed.
Property run_test_cleanup:
 A boolean indicating if test.cleanup should be run by default after a test completes, if the run_cleanup argument is not specified. [PERSISTENT]
Property num_tests_run:
 The number of tests run during the job. [OPTIONAL]
Property num_tests_failed:
 The number of tests failed during the job. [OPTIONAL]
Property bootloader:
 An instance of the boottool class. May not be available on job instances where access to the bootloader is not available (e.g. on the server running a server job). [OPTIONAL]
Property harness:
 An instance of the client test harness. Only available in contexts where client test execution happens. [OPTIONAL]
Property logging:
 An instance of the logging manager associated with the job.
Property profilers:
 An instance of the profiler manager associated with the job.
Property sysinfo:
 An instance of the sysinfo object. Only available in contexts where it’s possible to collect sysinfo.
Property warning_manager:
 A class for managing which types of WARN messages should be logged and which should be suppressed. [OPTIONAL]
Property warning_loggers:
 A set of readable streams that will be monitored for WARN messages to be logged. [OPTIONAL]
Abstract methods:
_find_base_directories [CLASSMETHOD]
Returns the location of autodir, clientdir and serverdir
_find_resultdir
Returns the location of resultdir. Gets a copy of any parameters passed into base_job.__init__. Can return None to indicate that no resultdir is to be used.
_get_status_logger
Returns a status_logger instance for recording job status logs.
autodir
automatic_test_tag
bindir
clientdir
configdir
conmuxdir
customtestdir
default_profile_only
get_state(name, default=<object object>)[source]

Returns the value associated with a particular name.

Parameters:
  • name – The name the value was saved with.
  • default – A default value to return if no state is currently associated with var.
Returns:

A deep copy of the value associated with name. Note that this explicitly returns a deep copy to avoid problems with mutable values; mutations are not persisted or shared.

Raises:

KeyError when no state is associated with var and a default value is not provided.

last_boot_tag
pkgdir
pop_execution_context()[source]

Reverse the effects of the previous push_execution_context call.

Raises:IndexError when the stack of contexts is empty.
profdir
push_execution_context(resultdir)[source]

Save off the current context of the job and change to the given one.

In practice method just changes the resultdir, but it may become more extensive in the future. The expected use case is for when a child job needs to be executed in some sort of nested context (for example the way parallel_simple does). The original context can be restored with a pop_execution_context call.

Parameters:resultdir – The new resultdir, relative to the current one.
record(status_code, subdir, operation, status='', optional_fields=None)[source]

Record a job-level status event.

Logs an event noteworthy to the Autotest job as a whole. Messages will be written into a global status log file, as well as a subdir-local status log file (if subdir is specified).

Parameters:
  • status_code – A string status code describing the type of status entry being recorded. It must pass log.is_valid_status to be considered valid.
  • subdir – A specific results subdirectory this also applies to, or None. If not None the subdirectory must exist.
  • operation – A string describing the operation that was run.
  • status – An optional human-readable message describing the status entry, for example an error message or “completed successfully”.
  • optional_fields – An optional dictionary of additional named fields to be included with the status message. Every time timestamp and localtime entries are generated with the current time and added to this dictionary.
record_entry(entry, log_in_subdir=True)[source]

Record a job-level status event, using a status_log_entry.

This is the same as self.record but using an existing status log entry object rather than constructing one for you.

Parameters:
  • entry – A status_log_entry object
  • log_in_subdir – A boolean that indicates (when true) that subdir logs should be written into the subdirectory status log file.
resultdir
run_test_cleanup
serverdir
set_state(name, value)[source]

Saves the value given with the provided name.

Parameters:
  • name – The name the value should be saved with.
  • value – The value to save.
site_testdir
tag
testdir
tmpdir
toolsdir
use_sequence_number
class autotest.client.shared.base_job.job_directory(path, is_writable=False)[source]

Bases: object

Represents a job.*dir directory.

exception JobDirectoryException[source]

Bases: autotest.client.shared.error.AutotestError

Generic job_directory exception superclass.

exception MissingDirectoryException(path)[source]

Bases: autotest.client.shared.base_job.JobDirectoryException

Raised when a directory required by the job does not exist.

exception UncreatableDirectoryException(path, error)[source]

Bases: autotest.client.shared.base_job.JobDirectoryException

Raised when a directory required by the job is missing and cannot be created.

exception UnwritableDirectoryException(path)[source]

Bases: autotest.client.shared.base_job.JobDirectoryException

Raised when a writable directory required by the job exists but is not writable.

static property_factory(attribute)[source]

Create a job.*dir -> job._*dir.path property accessor.

Parameters:attribute – A string with the name of the attribute this is exposed as. ‘_’+attribute must then be attribute that holds either None or a job_directory-like object
Returns:A read-only property object that exposes a job_directory path
class autotest.client.shared.base_job.job_state[source]

Bases: object

A class for managing explicit job and user state, optionally persistent.

The class allows you to save state by name (like a dictionary). Any state stored in this class should be picklable and deep copyable. While this is not enforced it is recommended that only valid python identifiers be used as names. Additionally, the namespace ‘stateful_property’ is used for storing the valued associated with properties constructed using the property_factory method.

NO_DEFAULT = <object object>
PICKLE_PROTOCOL = 2
discard(*args, **dargs)[source]

If namespace.name is a defined value, deletes it.

Parameters:
  • namespace (string) – The namespace that the property should be stored in.
  • name (string) – The name the value was saved with.
discard_namespace(*args, **dargs)[source]

Delete all defined namespace.* names.

Parameters:namespace (string) – The namespace to be cleared.
get(*args, **dargs)[source]

Returns the value associated with a particular name.

Parameters:
  • namespace (string) – The namespace that the property should be stored in.
  • name (string) – The name the value was saved with.
  • default (object) – A default value to return if no state is currently associated with var.
Returns:

A deep copy of the value associated with name. Note that this explicitly returns a deep copy to avoid problems with mutable values; mutations are not persisted or shared.

Raises:

KeyError raised when no state is associated with var and a default value is not provided.

has(*args, **dargs)[source]

Return a boolean indicating if namespace.name is defined.

Parameters:
  • namespace (string) – The namespace that the property should be stored in.
  • name (string) – The name the value was saved with.
Returns:

True if the given name is defined in the given namespace and False otherwise.

Return type:

bool

static property_factory(state_attribute, property_attribute, default, namespace='global_properties')[source]

Create a property object for an attribute using self.get and self.set.

Parameters:
  • state_attribute – A string with the name of the attribute on job that contains the job_state instance.
  • property_attribute – A string with the name of the attribute this property is exposed as.
  • default – A default value that should be used for this property if it is not set.
  • namespace – The namespace to store the attribute value in.
Returns:

A read-write property object that performs self.get calls to read the value and self.set calls to set it.

read_from_file(file_path, merge=True)[source]

Read in any state from the file at file_path.

When merge=True, any state specified only in-memory will be preserved. Any state specified on-disk will be set in-memory, even if an in-memory setting already exists.

Parameters:
  • file_path (string) – The path where the state should be read from. It must exist but it can be empty.
  • merge (bool) – If true, merge the on-disk state with the in-memory state. If false, replace the in-memory state with the on-disk state.

Warning: This method is intentionally concurrency-unsafe. It makes no attempt to control concurrent access to the file at file_path.

set(*args, **dargs)[source]

Saves the value given with the provided name.

Parameters:
  • namespace (string) – The namespace that the property should be stored in.
  • name (string) – The name the value was saved with.
  • value – The value to save.
set_backing_file(file_path)[source]

Change the path used as the backing file for the persistent state.

When a new backing file is specified if a file already exists then its contents will be added into the current state, with conflicts between the file and memory being resolved in favor of the file contents. The file will then be kept in sync with the (combined) in-memory state. The syncing can be disabled by setting this to None.

Parameters:file_path (string) – A path on the filesystem that can be read from and written to, or None to turn off the backing store.
write_to_file(file_path)[source]

Write out the current state to the given path.

Warning: This method is intentionally concurrency-unsafe. It makes no attempt to control concurrent access to the file at file_path.

Parameters:file_path (string) – The path where the state should be written out to. Must be writable.
class autotest.client.shared.base_job.status_indenter[source]

Bases: object

Abstract interface that a status log indenter should use.

decrement()[source]

Decrease indentation by one level.

increment()[source]

Increase indentation by one level.

indent
class autotest.client.shared.base_job.status_log_entry(status_code, subdir, operation, message, fields, timestamp=None)[source]

Bases: object

Represents a single status log entry.

BAD_CHAR_REGEX = <_sre.SRE_Pattern object>
LOCALTIME_FIELD = 'localtime'
RENDERED_NONE_VALUE = '----'
TIMESTAMP_FIELD = 'timestamp'
is_end()[source]

Indicates if this status log is the end of a nested block.

Returns:A boolean indicating if this entry ends a nested block.
is_start()[source]

Indicates if this status log is the start of a new nested block.

Returns:A boolean indicating if this entry starts a new nested block.
classmethod parse(line)[source]

Parse a status log entry from a text string.

This method is the inverse of render; it should always be true that parse(entry.render()) produces a new status_log_entry equivalent to entry.

Returns:A new status_log_entry instance with fields extracted from the given status line. If the line is an extra message line then None is returned.
render()[source]

Render the status log entry into a text string.

Returns:A text string suitable for writing into a status log file.
class autotest.client.shared.base_job.status_logger(job, indenter, global_filename='status', subdir_filename='status', record_hook=None, tap_writer=None)[source]

Bases: object

Represents a status log file. Responsible for translating messages into on-disk status log lines.

Property global_filename:
 The filename to write top-level logs to.
Property subdir_filename:
 The filename to write subdir-level logs to.
record_entry(log_entry, log_in_subdir=True)[source]

Record a status_log_entry into the appropriate status log files.

Parameters:
  • log_entry – A status_log_entry instance to be recorded into the status logs.
  • log_in_subdir – A boolean that indicates (when true) that subdir logs should be written into the subdirectory status log file.
render_entry(log_entry)[source]

Render a status_log_entry as it would be written to a log file.

Parameters:log_entry – A status_log_entry instance to be rendered.
Returns:The status log entry, rendered as it would be written to the logs (including indentation).
autotest.client.shared.base_job.with_backing_file(method)[source]

A decorator to perform a lock-read-*-write-unlock cycle.

When applied to a method, this decorator will automatically wrap calls to the method in a lock-and-read before the call followed by a write-and-unlock. Any operation that is reading or writing state should be decorated with this method to ensure that backing file state is consistently maintained.

autotest.client.shared.base_job.with_backing_lock(method)[source]

A decorator to perform a lock-*-unlock cycle.

When applied to a method, this decorator will automatically wrap calls to the method in a backing file lock and before the call followed by a backing file unlock.

base_packages Module

This module defines the BasePackageManager Class which provides an implementation of the packaging system API providing methods to fetch, upload and remove packages. Site specific extensions to any of these methods should inherit this class.

class autotest.client.shared.base_packages.BasePackageManager(pkgmgr_dir, hostname=None, repo_urls=None, upload_paths=None, do_locking=True, run_function=<function run>, run_function_args=[], run_function_dargs={})[source]

Bases: object

add_repository(repo)[source]
compare_checksum(pkg_path, repo_url)[source]

Calculate the checksum of the file specified in pkg_path and compare it with the checksum in the checksum file Return True if both match else return False. :param pkg_path: The full path to the package file for which the checksum is being compared :param repo_url: The URL to fetch the checksum from

compute_checksum(pkg_path)[source]

Compute the MD5 checksum for the package file and return it. pkg_path : The complete path for the package file

fetch_pkg(pkg_name, dest_path, repo_url=None, use_checksum=False, install=False)[source]

Fetch the package into dest_dir from repo_url. By default repo_url is None and the package is looked in all the repositories specified. Otherwise it fetches it from the specific repo_url. pkg_name : name of the package (ex: test-sleeptest.tar.bz2,

dep-gcc.tar.bz2, kernel.1-1.rpm)

repo_url : the URL of the repository where the package is located. dest_path : complete path of where the package will be fetched to. use_checksum : This is set to False to fetch the packages.checksum file

so that the checksum comparison is bypassed for the checksum file itself. This is used internally by the packaging system. It should be ignored by externals callers of this method who use it fetch custom packages.
install : install path has unique name and destination requirements
that vary based on the fetcher that is used. So call them here as opposed to install_pkg.
get_fetcher(url)[source]
get_mirror_list(repo_urls)[source]

Stub function for site specific mirrors.

Returns:
Priority ordered list
get_package_name(url, pkg_type)[source]

Extract the group and test name for the url. This method is currently used only for tests.

static get_tarball_name(name, pkg_type)[source]

Converts a package name and type into a tarball name.

Parameters:
  • name – The name of the package
  • pkg_type – The type of the package
Returns:

A tarball filename for that specific type of package

install_pkg(name, pkg_type, fetch_dir, install_dir, preserve_install_dir=False, repo_url=None)[source]

Remove install_dir if it already exists and then recreate it unless preserve_install_dir is specified as True. Fetch the package into the pkg_dir. Untar the package into install_dir The assumption is that packages are of the form : <pkg_type>.<pkg_name>.tar.bz2 name : name of the package type : type of the package fetch_dir : The directory into which the package tarball will be

fetched to.

install_dir : the directory where the package files will be untarred to repo_url : the url of the repository to fetch the package from.

static parse_tarball_name(tarball_name)[source]

Coverts a package tarball name into a package name and type.

Parameters:tarball_name – The filename of the tarball
Returns:(name, pkg_type) where name is the package name and pkg_type is the package type.
remove_checksum(pkg_name)[source]

Remove the checksum of the package from the packages checksum file. This method is called whenever a package is removed from the repositories in order clean its corresponding checksum. pkg_name : The name of the package to be removed

remove_pkg(pkg_name, remove_path=None, remove_checksum=False)[source]

Remove the package from the specified remove_path pkg_name : name of the package (ex: test-sleeptest.tar.bz2,

dep-gcc.tar.bz2)

remove_path : the location to remove the package from.

remove_pkg_file(filename, pkg_dir)[source]

Remove the file named filename from pkg_dir

repo_check(repo)[source]

Check to make sure the repo is in a sane state: ensure we have at least XX amount of free space Make sure we can write to the repo

tar_package(pkg_name, src_dir, dest_dir, include_string=None, exclude_string=None)[source]

Create a tar.bz2 file with the name ‘pkg_name’ say test-blah.tar.bz2.

Includes the files specified in include_string, and excludes the files specified on the exclude string, while tarring the source. Returns the destination tarball path.

Parameters:
  • pkg_name – Package name.
  • src_dir – Directory that contains the data to be packaged.
  • dest_dir – Directory that will hold the destination tarball.
  • include_string – Pattern that represents the files that will be added to the tar package.
  • exclude_string – Pattern that represents the files that should be excluded from the tar package. It could be either a string or a list.
untar_pkg(tarball_path, dest_dir)[source]

Untar the package present in the tarball_path and put a “.checksum” file in the dest_dir containing the checksum of the tarball. This method assumes that the package to be untarred is of the form <name>.tar.bz2

untar_required(tarball_path, dest_dir)[source]

Compare the checksum of the tarball_path with the .checksum file in the dest_dir and return False if it matches. The untar of the package happens only if the checksums do not match.

update_checksum(pkg_path)[source]

Update the checksum of the package in the packages’ checksum file. This method is called whenever a package is fetched just to be sure that the checksums in the local file are the latest. pkg_path : The complete path to the package file.

upkeep(custom_repos=None)[source]

Clean up custom upload/download areas

upload_pkg(pkg_path, upload_path=None, update_checksum=False, timeout=300)[source]
upload_pkg_dir(dir_path, upload_path)[source]

Upload a full directory. Depending on the upload path, the appropriate method for that protocol is called. Currently this copies the whole tmp package directory to the target directory. This assumes that the web server is running on the same machine where the method is being called from. The upload_path’s files are basically served by that web server.

upload_pkg_file(file_path, upload_path)[source]

Upload a single file. Depending on the upload path, the appropriate method for that protocol is called. Currently this simply copies the file to the target directory (but can be extended for other protocols) This assumes that the web server is running on the same machine where the method is being called from. The upload_path’s files are basically served by that web server.

upload_pkg_parallel(pkg_path, upload_path, update_checksum=False)[source]

Uploads to a specified upload_path or to all the repos. Also uploads the checksum file to all the repos. pkg_path : The complete path to the package file upload_path : the absolute path where the files are copied to.

if set to ‘None’ assumes ‘all’ repos
update_checksum : If set to False, the checksum file is not
going to be updated which happens by default. This is necessary for custom packages (like custom kernels and custom tests) that get uploaded which do not need to be part of the checksum file and bloat it.
class autotest.client.shared.base_packages.GitFetcher(package_manager, repository_url)[source]

Bases: autotest.client.shared.base_packages.RepositoryFetcher

A git based repository fetcher

fetch_pkg_file(filename, dest_path)[source]

Fetch a package file and save it to the given destination path

git is an SCM, you can download the test directly. No need to fetch a bz2’d tarball file. However ‘filename’ is <type>-<name>.tar.bz2 break this up and only fetch <name>.

Parameters:
  • filename (string) – The filename of the package file to fetch.
  • dest_path (string) – Destination path to download the file to.
git_archive_cmd_pattern = 'git archive --remote=%s -o %s %s'
install_pkg_post(filename, fetch_dir, install_dir, preserve_install_dir=False)[source]

Fetcher specific post install

Parameters:
  • filename (string) – The filename of the package to install
  • fetch_dir (string) – The fetched path of the package
  • install_dir (string) – The path to install the package to

@preserve_install_dir: Preserve the install directory

class autotest.client.shared.base_packages.HttpFetcher(package_manager, repository_url)[source]

Bases: autotest.client.shared.base_packages.RepositoryFetcher

Repository Fetcher using HTTP

fetch_pkg_file(filename, dest_path)[source]

Fetch a package file from a package repository.

Parameters:
  • filename (string) – The filename of the package file to fetch.
  • dest_path (string) – Destination path to download the file to.
Raises:

PackageFetchError – if the fetch failed

wget_cmd_pattern = 'wget --connect-timeout=15 -nv %s -O %s'
class autotest.client.shared.base_packages.LocalFilesystemFetcher(package_manager, repository_url)[source]

Bases: autotest.client.shared.base_packages.RepositoryFetcher

fetch_pkg_file(filename, dest_path)[source]

Fetch a package file from a package repository.

Parameters:
  • filename (string) – The filename of the package file to fetch.
  • dest_path (string) – Destination path to download the file to.
Raises:

PackageFetchError – if the fetch failed

class autotest.client.shared.base_packages.RepositoryFetcher(package_manager, repository_url)[source]

Bases: object

Base class with common functionality for repository fetchers

fetch_pkg_file(filename, dest_path)[source]

Fetch a package file from a package repository.

Parameters:
  • filename (string) – The filename of the package file to fetch.
  • dest_path (string) – Destination path to download the file to.
Raises:

PackageFetchError – if the fetch failed

install_pkg_post(filename, fetch_dir, install_dir, preserve_install_dir=False)[source]

Fetcher specific post install

Parameters:
  • filename (string) – The filename of the package to install
  • fetch_dir (string) – The fetched path of the package
  • install_dir (string) – The path to install the package to

@preserve_install_dir: Preserve the install directory

install_pkg_setup(name, fetch_dir, install)[source]

Install setup for a package based on fetcher type.

Parameters:
  • name (string) – The filename to be munged
  • fetch_dir (string) – The destination path to be munged
  • install (boolean) – Whether this is be called from the install path or not
Returns:

tuple with (name, fetch_dir)

url = None
autotest.client.shared.base_packages.check_diskspace(repo, min_free=None)[source]

Check if the remote directory over at the pkg repo has available diskspace

If the amount of free space is not supplied, it is taken from the global configuration file, section [PACKAGES], key ‘mininum_free_space’. The unit used are in SI, that is, 1 GB = 10**9 bytes.

Parameters:

repo (string) – a remote package repo URL

Param:

min_free mininum amount of free space, in GB (10**9 bytes)

Raises:
autotest.client.shared.base_packages.check_write(repo)[source]

Checks that the remote repository directory is writable

Parameters:repo (string) – a remote package repo URL
Raises:error.RepoWriteError – repository write error
autotest.client.shared.base_packages.create_directory(repo)[source]

Create a directory over at the remote repository

Parameters:repo (string) – the repo URL containing the remote directory path
Returns:a CmdResult object or None
autotest.client.shared.base_packages.has_pbzip2()[source]

Check if parallel bzip2 is available on this system.

Returns:True if pbzip2 is available, False otherwise
autotest.client.shared.base_packages.parse_ssh_path(repo)[source]

Parse an SSH url

Parameters:repo (string) – a repo uri like ssh://xx@xx/path/to/
Returns:tuple with (host, remote_path)
autotest.client.shared.base_packages.repo_run_command(repo, cmd, ignore_status=False, cd=True)[source]

Run a command relative to the repo path

This is basically a utils.run() wrapper that sets itself in a repo directory if it is appropriate, so parameters such as cmd and ignore_status are passed along to it.

Parameters:
  • repo (string) – a repository url
  • cmd (string) – the command to be executed. This is passed along to utils.run()
  • ignore_status (boolean) – do not raise an exception, no matter what the exit code of the command is.
  • cd (boolean) – wether to change the working directory to the repo directory before running the specified command.
Returns:

a CmdResult object or None

Raises:

CmdError – the exit code of the command execution was not 0

autotest.client.shared.base_packages.trim_custom_directories(repo, older_than_days=None)[source]

Remove old files from the remote repo directory

The age of the files, if not provided by the older_than_days parameter is taken from the global configuration file, at section [PACKAGES], configuration item ‘custom_max_age’.

Parameters:repo (string) – a remote package repo URL

base_syncdata Module

class autotest.client.shared.base_syncdata.SessionData(hosts, timeout)[source]

Bases: object

close()[source]
is_finished()[source]
set_finish()[source]
timeout()[source]
class autotest.client.shared.base_syncdata.SyncData(masterid, hostid, hosts, session_id=None, listen_server=None, port=13234, tmpdir=None)[source]

Bases: object

Provides data synchronization between hosts.

Transferred data is pickled and sent to all destination points. If there is no listen server it will create a new one. If multiple hosts wants to communicate with each other, then communications are identified by session_id.

close()[source]
single_sync(data=None, timeout=60, session_id=None)[source]
sync(data=None, timeout=60, session_id=None)[source]

Synchronize data between hosts.

timeout()[source]
class autotest.client.shared.base_syncdata.SyncListenServer(address='', port=13234, tmpdir=None)[source]

Bases: object

close()[source]

Close SyncListenServer thread.

Close all open connection with clients and listen server.

class autotest.client.shared.base_syncdata.TempDir(tmpdir=None)[source]

Bases: autotest.client.shared.autotemp.tempdir

TempDir class is tempdir for predefined tmpdir.

clean()[source]

Should not delete predefined tmpdir.

autotest.client.shared.base_syncdata.net_recv_object(sock, timeout=60)[source]

Receive python object over network.

Parameters:
  • ip_addr – ipaddres of waiter for data.
  • obj – object to send
Returns:

object from network

autotest.client.shared.base_syncdata.net_send_object(sock, obj)[source]

Send python object over network.

Parameters:
  • ip_addr – ipaddres of waiter for data.
  • obj – object to send

boottool Module

boottool client-side module.

This module provides an API for client side tests that need to manipulate boot entries. It’s based on the rewrite of boottool, now python and grubby based. It aims to be keep API compatibility with the older version, except from XEN support which has been removed. We’ll gladly accept patches that provide full coverage for this mode/feature.

Copyright 2009 Google Inc. Copyright 2012 Red Hat, Inc.

Released under the GPL v2

class autotest.client.shared.boottool.boottool(path=None)[source]

Bases: autotest.client.tools.boottool.Grubby

Client site side boottool wrapper.

Inherits all functionality from boottool(.py) CLI app (lazily).

check_version Module

class autotest.client.shared.check_version.check_python_version[source]

Bases: autotest.client.shared.check_version.site_check_python_version, autotest.client.shared.base_check_version.base_check_python_version

class autotest.client.shared.check_version.site_check_python_version[source]

common Module

control_data Module

class autotest.client.shared.control_data.ControlData(vars, path, raise_warnings=False)[source]

Bases: object

set_attr(attr, val, raise_warnings=False)[source]
set_author(val)[source]
set_dependencies(val)[source]
set_doc(val)[source]
set_experimental(val)[source]
set_name(val)[source]
set_run_verify(val)[source]
set_sync_count(val)[source]
set_test_category(val)[source]
set_test_class(val)[source]
set_test_parameters(val)[source]
set_test_type(val)[source]
set_time(val)[source]
exception autotest.client.shared.control_data.ControlVariableException[source]

Bases: exceptions.Exception

autotest.client.shared.control_data.parse_control(path, raise_warnings=False)[source]

distro Module

This module provides the client facilities to detect the Linux Distribution it’s running under.

This is a replacement for the get_os_vendor() function from the utils module.

class autotest.client.shared.distro.LinuxDistro(name, version, release, arch)[source]

Bases: object

Simple collection of information for a Linux Distribution

class autotest.client.shared.distro.Probe[source]

Bases: object

Probes the machine and does it best to confirm it’s the right distro

CHECK_FILE = None

Points to a file that can determine if this machine is running a given Linux Distribution. This servers a first check that enables the extra checks to carry on.

CHECK_FILE_CONTAINS = None

Sets the content that should be checked on the file pointed to by CHECK_FILE_EXISTS. Leave it set to None (its default) to check only if the file exists, and not check its contents

CHECK_FILE_DISTRO_NAME = None

The name of the Linux Distribution to be returned if the file defined by CHECK_FILE_EXISTS exist.

CHECK_VERSION_REGEX = None

A regular expresion that will be run on the file pointed to by CHECK_FILE_EXISTS

check_name_for_file()[source]

Checks if this class will look for a file and return a distro

The conditions that must be true include the file that identifies the distro file being set (CHECK_FILE) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME)

check_name_for_file_contains()[source]

Checks if this class will look for text on a file and return a distro

The conditions that must be true include the file that identifies the distro file being set (CHECK_FILE), the text to look for inside the distro file (CHECK_FILE_CONTAINS) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME)

check_release()[source]

Checks if this has the conditions met to look for the release number

check_version()[source]

Checks if this class will look for a regex in file and return a distro

get_distro()[source]

Returns the LinuxDistro this probe detected

name_for_file()[source]

Get the distro name if the CHECK_FILE is set and exists

name_for_file_contains()[source]

Get the distro if the CHECK_FILE is set and has content

release()[source]

Returns the release of the distro

version()[source]

Returns the version of the distro

autotest.client.shared.distro.register_probe(probe_class)[source]

Register a probe to be run during autodetection

autotest.client.shared.distro.detect()[source]

Attempts to detect the Linux Distribution running on this machine

Returns:the detected LinuxDistro or UNKNOWN_DISTRO
Return type:LinuxDistro

distro_def Module

This module defines a structure and portable format for relevant information on Linux Distributions in such a way that information about known distros can be packed and distributed.

Please note that this module deals with Linux Distributions not necessarily installed on the running system.

autotest.client.shared.distro_def.save(linux_distro, path)[source]

Saves the linux_distro to an external file format

Parameters:
Returns:

None

autotest.client.shared.distro_def.load(path)[source]

Loads the distro from an external file

Parameters:path (str) – the location for the input file
Returns:an DistroDef instance
Return type:DistroDef
autotest.client.shared.distro_def.load_from_tree(name, version, release, arch, package_type, path)[source]

Loads a DistroDef from an installable tree

Parameters:
  • name (str) – a short name that precisely distinguishes this Linux Distribution among all others.
  • version (str) – the major version of the distribution. Usually this is a single number that denotes a large development cycle and support file.
  • release (str) – the release or minor version of the distribution. Usually this is also a single number, that is often omitted or starts with a 0 when the major version is initially release. It’s ofter associated with a shorter development cycle that contains incremental a collection of improvements and fixes.
  • arch (str) – the main target for this Linux Distribution. It’s common for some architectures to ship with packages for previous and still compatible architectures, such as it’s the case with Intel/AMD 64 bit architecture that support 32 bit code. In cases like this, this should be set to the 64 bit architecture name.
  • package_type (str) – one of the available package info loader types
  • path (str) – top level directory of the distro installation tree files
class autotest.client.shared.distro_def.SoftwarePackage(name, version, release, checksum, arch)[source]

Bases: object

Definition of relevant information on a software package

class autotest.client.shared.distro_def.DistroDef(name, version, release, arch)[source]

Bases: autotest.client.shared.distro.LinuxDistro

More complete information on a given Linux Distribution

software_packages = None

All the software packages that ship with this Linux distro

software_packages_type = None

A simple text that denotes the software type that makes this distro

autotest.client.shared.distro_def.DISTRO_PKG_INFO_LOADERS = {'deb': <class 'autotest.client.shared.distro_def.DistroPkgInfoLoaderDeb'>, 'rpm': <class 'autotest.client.shared.distro_def.DistroPkgInfoLoaderRpm'>}

the type of distro that will determine what loader will be used

enum Module

Generic enumeration support.

class autotest.client.shared.enum.Enum(*names, **kwargs)[source]

Bases: object

Utility class to implement Enum-like functionality.

>>> e = Enum('String one', 'String two')
>>> e.STRING_ONE
0
>>> e.STRING_TWO
1
>>> e.choices()
[(0, 'String one'), (1, 'String two')]
>>> e.get_value('String one')
0
>>> e.get_string(0)
'String one'
>>> e = Enum('Hello', 'Goodbye', string_values=True)
>>> e.HELLO, e.GOODBYE
('Hello', 'Goodbye')
>>> e = Enum('One', 'Two', start_value=1)
>>> e.ONE
1
>>> e.TWO
2
choices()[source]

Return choice list suitable for Django model choices.

static get_attr_name(string)[source]
get_string(value)[source]

Given a value, get the string name for it.

get_value(name)[source]

Convert a string name to it’s corresponding value. If a value is passed in, it is returned.

error Module

Internal global error types

autotest.client.shared.error.format_error()[source]
autotest.client.shared.error.context_aware(fn)[source]

A decorator that must be applied to functions that call context().

autotest.client.shared.error.context(s='', log=None)[source]

Set the context for the currently executing function and optionally log it.

Parameters:
  • s – A string. If not provided, the context for the current function will be cleared.
  • log – A logging function to pass the context message to. If None, no function will be called.
autotest.client.shared.error.get_context()[source]

Return the current context (or None if none is defined).

autotest.client.shared.error.exception_context(e)[source]

Return the context of a given exception (or None if none is defined).

exception autotest.client.shared.error.AutoservHostIsShuttingDownError[source]

Bases: autotest.client.shared.error.AutoservHostError

Host is shutting down

exception autotest.client.shared.error.TestBug[source]

Bases: autotest.client.shared.error.TestBaseException

Indicates that the test failed, but the fail was expected.

exit_status = 'BUG'
exception autotest.client.shared.error.AutoservHardwareRepairRequiredError[source]

Bases: autotest.client.shared.error.AutoservError

Exception class raised during repairs to indicate that a hardware repair is going to be necessary.

exception autotest.client.shared.error.RepoWriteError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when packager cannot write to a repo’s desitnation

exception autotest.client.shared.error.AutoservUnsupportedError[source]

Bases: autotest.client.shared.error.AutoservError

Error raised when you try to use an unsupported optional feature

exception autotest.client.shared.error.CmdError(command, result_obj, additional_text=None)[source]

Bases: autotest.client.shared.error.TestError

Indicates that a command failed, is fatal to the test unless caught.

exception autotest.client.shared.error.AutotestError[source]

Bases: exceptions.Exception

The parent of all errors deliberately thrown within the client code.

exception autotest.client.shared.error.RepoDiskFullError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when the destination for packages is full

exception autotest.client.shared.error.AutoservRebootError[source]

Bases: autotest.client.shared.error.AutoservError

Error occurred while rebooting a machine

exception autotest.client.shared.error.TestWarn[source]

Bases: autotest.client.shared.error.TestBaseException

Indicates that bad things (may) have happened, but not an explicit failure.

exit_status = 'WARN'
exception autotest.client.shared.error.PackageInstallError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when there is an error installing the package

exception autotest.client.shared.error.HostInstallProfileError[source]

Bases: autotest.client.shared.error.JobError

Indicates the machine failed to have a profile assigned.

exception autotest.client.shared.error.PackageError[source]

Bases: autotest.client.shared.error.TestError

Indicates an error trying to perform a package operation.

exception autotest.client.shared.error.AutotestHostRunError(description, result_obj)[source]

Bases: autotest.client.shared.error.HostRunErrorMixIn, autotest.client.shared.error.AutotestError

exception autotest.client.shared.error.UnhandledTestFail(unhandled_exception)[source]

Bases: autotest.client.shared.error.TestFail

Indicates an unhandled fail in a test.

exception autotest.client.shared.error.BarrierAbortError[source]

Bases: autotest.client.shared.error.BarrierError

Indicate that the barrier was explicitly aborted by a member.

exception autotest.client.shared.error.AutoservSubcommandError(func, exit_code)[source]

Bases: autotest.client.shared.error.AutoservError

Indicates an error while executing a (forked) subcommand

exception autotest.client.shared.error.NetCommunicationError[source]

Bases: autotest.client.shared.error.JobError

Indicate that network communication was broken.

exception autotest.client.shared.error.AutoservShutdownError[source]

Bases: autotest.client.shared.error.AutoservRebootError

Error occurred during shutdown of machine

exception autotest.client.shared.error.PackageRemoveError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when there is an error removing the package

exception autotest.client.shared.error.UnhandledTestError(unhandled_exception)[source]

Bases: autotest.client.shared.error.TestError

Indicates an unhandled error in a test.

exception autotest.client.shared.error.DataSyncError[source]

Bases: autotest.client.shared.error.NetCommunicationError

Indicates problem during synchronization data over network.

exception autotest.client.shared.error.AutoservHostError[source]

Bases: autotest.client.shared.error.AutoservError

Error reaching a host

exception autotest.client.shared.error.TestBaseException[source]

Bases: autotest.client.shared.error.AutotestError

The parent of all test exceptions.

exit_status = 'NEVER_RAISE_THIS'
exception autotest.client.shared.error.TestNAError[source]

Bases: autotest.client.shared.error.TestBaseException

Indictates that the test is Not Applicable. Should be thrown when various conditions are such that the test is inappropriate.

exit_status = 'TEST_NA'
exception autotest.client.shared.error.AutoservHardwareHostError[source]

Bases: autotest.client.shared.error.AutoservHostError

Found hardware problems with the host

exception autotest.client.shared.error.AutoservError[source]

Bases: exceptions.Exception

exception autotest.client.shared.error.AutoservSSHTimeout[source]

Bases: autotest.client.shared.error.AutoservError

SSH experienced a connection timeout

exception autotest.client.shared.error.InstallError[source]

Bases: autotest.client.shared.error.JobError

Indicates an installation error which Terminates and fails the job.

exception autotest.client.shared.error.AutoservDiskFullHostError(path, want_gb, free_space_gb)[source]

Bases: autotest.client.shared.error.AutoservHostError

Not enough free disk space on host

exception autotest.client.shared.error.AutoservInstallError[source]

Bases: autotest.client.shared.error.AutoservError

Error occurred while installing autotest on a host

exception autotest.client.shared.error.TestError[source]

Bases: autotest.client.shared.error.TestBaseException

Indicates that something went wrong with the test harness itself.

exit_status = 'ERROR'
exception autotest.client.shared.error.AutoservVirtError[source]

Bases: autotest.client.shared.error.AutoservError

Vitualization related error

exception autotest.client.shared.error.BarrierError[source]

Bases: autotest.client.shared.error.JobError

Indicates an error happened during a barrier operation.

exception autotest.client.shared.error.AutotestRunError[source]

Bases: autotest.client.shared.error.AutotestError

Indicates a problem running server side control files.

exception autotest.client.shared.error.RepoError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when a repo isn’t working in some way

exception autotest.client.shared.error.PackagingError[source]

Bases: autotest.client.shared.error.AutotestError

Abstract error class for all packaging related errors.

exception autotest.client.shared.error.RepoUnknownError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when packager cannot write to a repo’s desitnation

exception autotest.client.shared.error.UnhandledJobError(unhandled_exception)[source]

Bases: autotest.client.shared.error.JobError

Indicates an unhandled error in a job.

exception autotest.client.shared.error.TestFail[source]

Bases: autotest.client.shared.error.TestBaseException

Indicates that the test failed, but the job will not continue.

exit_status = 'FAIL'
exception autotest.client.shared.error.JobError[source]

Bases: autotest.client.shared.error.AutotestError

Indicates an error which terminates and fails the whole job (ABORT).

exception autotest.client.shared.error.AutoservRunError(description, result_obj)[source]

Bases: autotest.client.shared.error.HostRunErrorMixIn, autotest.client.shared.error.AutoservError

exception autotest.client.shared.error.PackageFetchError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when there is an error fetching the package

exception autotest.client.shared.error.PackageUploadError[source]

Bases: autotest.client.shared.error.PackagingError

Raised when there is an error uploading the package

exception autotest.client.shared.error.AutoservHardwareRepairRequestedError[source]

Bases: autotest.client.shared.error.AutoservError

Exception class raised from Host.repair_full() (or overrides) when software repair fails but it successfully managed to request a hardware repair (by notifying the staff, sending mail, etc)

exception autotest.client.shared.error.HostRunErrorMixIn(description, result_obj)[source]

Bases: exceptions.Exception

Indicates a problem in the host run() function raised from client code. Should always be constructed with a tuple of two args (error description (str), run result object). This is a common class mixed in to create the client and server side versions of it.

exception autotest.client.shared.error.HarnessError[source]

Bases: autotest.client.shared.error.JobError

Indicates problem with the harness.

exception autotest.client.shared.error.AutoservNotMountedHostError[source]

Bases: autotest.client.shared.error.AutoservHostError

Found unmounted partitions that should be mounted

exception autotest.client.shared.error.AutoservSshPermissionDeniedError(description, result_obj)[source]

Bases: autotest.client.shared.error.AutoservRunError

Indicates that a SSH permission denied error was encountered.

exception autotest.client.shared.error.HostInstallTimeoutError[source]

Bases: autotest.client.shared.error.JobError

Indicates the machine failed to be installed after the predetermined timeout.

exception autotest.client.shared.error.AutoservSshPingHostError[source]

Bases: autotest.client.shared.error.AutoservHostError

SSH ping failed

exception autotest.client.shared.error.AutotestTimeoutError[source]

Bases: autotest.client.shared.error.AutotestError

This exception is raised when an autotest test exceeds the timeout parameter passed to run_timed_test and is killed.

git Module

Code that helps to deal with content from git repositories

class autotest.client.shared.git.GitRepoHelper(uri, branch='master', lbranch=None, commit=None, destination_dir=None, base_uri=None)[source]

Bases: object

Helps to deal with git repos, mostly fetching content from a repo

checkout(branch=None, commit=None)[source]

Performs a git checkout for a given branch and start point (commit)

Parameters:
  • branch – Remote branch name.
  • commit – Specific commit hash.
execute()[source]

Performs all steps necessary to initialize and download a git repo.

This includes the init, fetch and checkout steps in one single utility method.

fetch(uri)[source]

Performs a git fetch from the remote repo

get_top_commit()[source]

Returns the topmost commit id for the current branch.

Returns:Commit id.
get_top_tag()[source]

Returns the topmost tag for the current branch.

Returns:Tag.
git_cmd(cmd, ignore_status=False)[source]

Wraps git commands.

Parameters:
  • cmd – Command to be executed.
  • ignore_status – Whether we should suppress error.CmdError exceptions if the command did return exit code !=0 (True), or not suppress them (False).
init()[source]

Initializes a directory for receiving a verbatim copy of git repo

This creates a directory if necessary, and either resets or inits the repo

autotest.client.shared.git.get_repo(uri, branch='master', lbranch=None, commit=None, destination_dir=None, base_uri=None)[source]

Utility function that retrieves a given git code repository.

Parameters:
  • uri (string) – git repository url
  • branch (string) – git remote branch
  • destination_dir (string) – path of a dir where to save downloaded code
  • commit (string) – specific commit to download
  • lbranch (string) – git local branch name, if different from remote
  • uri – a closer, usually local, git repository url from where to fetch content first from

host_protections Module

host_queue_entry_states Module

This module contains the status enums for use by HostQueueEntrys in the database. It is a stand alone module as these status strings are needed from various disconnected pieces of code that should not depend on everything that autotest.frontend.afe.models depends on such as RPC clients.

iscsi Module

Basic iscsi support for Linux host with the help of commands iscsiadm and tgtadm.

This include the basic operates such as login and get device name by target name. And it can support the real iscsi access and emulated iscsi in localhost then access it.

class autotest.client.shared.iscsi.Iscsi(params, root_dir='/tmp')[source]

Bases: object

Basic iscsi support class. Will handle the emulated iscsi export and access to both real iscsi and emulated iscsi device.

cleanup()[source]

Clean up env after iscsi used.

delete_target()[source]

Delete target from host.

export_target()[source]

Export target in localhost for emulated iscsi

get_device_name()[source]

Get device name from the target name.

get_target_id()[source]

Get target id from image name. Only works for emulated iscsi device

logged_in()[source]

Check if the session is login or not.

login()[source]

Login session for both real iscsi device and emulated iscsi. Include env check and setup.

logout()[source]

Logout from target.

portal_visible()[source]

Check if the portal can be found or not.

autotest.client.shared.iscsi.iscsi_discover(portal_ip)[source]

Query from iscsi server for available targets

Parameters:portal_ip – Ip for iscsi server
autotest.client.shared.iscsi.iscsi_get_nodes()[source]

Get the iscsi nodes

autotest.client.shared.iscsi.iscsi_get_sessions()[source]

Get the iscsi sessions activated

autotest.client.shared.iscsi.iscsi_login(target_name)[source]

Login to a target with the target name

Parameters:target_name – Name of the target
autotest.client.shared.iscsi.iscsi_logout(target_name=None)[source]

Logout from a target. If the target name is not set then logout all targets.

Params target_name:
 Name of the target.

iso9660 Module

Basic ISO9660 file-system support.

This code does not attempt (so far) to implement code that knows about ISO9660 internal structure. Instead, it uses commonly available support either in userspace tools or on the Linux kernel itself (via mount).

autotest.client.shared.iso9660.iso9660(path)[source]

Checks the avaiable tools on a system and chooses class accordingly

This is a convinience function, that will pick the first avaialable iso9660 capable tool.

Parameters:path (str) – path to an iso9660 image file
Returns:an instance of any iso9660 capable tool
Return type:Iso9660IsoInfo, Iso9660IsoRead, Iso9660Mount or None
class autotest.client.shared.iso9660.Iso9660IsoInfo(path)[source]

Bases: autotest.client.shared.iso9660.BaseIso9660

Represents a ISO9660 filesystem

This implementation is based on the cdrkit’s isoinfo tool

read(path)[source]

Abstract method to read data from path

Parameters:path – path to the file
Returns:data content from the file
Return type:str
class autotest.client.shared.iso9660.Iso9660IsoRead(path)[source]

Bases: autotest.client.shared.iso9660.BaseIso9660

Represents a ISO9660 filesystem

This implementation is based on the libcdio’s iso-read tool

close()[source]

Cleanup and free any resources being used

Return type:None
copy(src, dst)[source]

Simplistic version of copy that relies on read()

Parameters:
  • src (str) – source path
  • dst (str) – destination path
Return type:

None

read(path)[source]

Abstract method to read data from path

Parameters:path – path to the file
Returns:data content from the file
Return type:str
class autotest.client.shared.iso9660.Iso9660Mount(path)[source]

Bases: autotest.client.shared.iso9660.BaseIso9660

Represents a mounted ISO9660 filesystem.

close()[source]

Perform umount operation on the temporary dir

Return type:None
copy(src, dst)[source]
Parameters:
  • src (str) – source
  • dst (str) – destination
Return type:

None

read(path)[source]

Read data from path

Parameters:path (str) – path to read data
Returns:data content
Return type:str

jsontemplate Module

Python implementation of json-template.

JSON Template is a minimal and powerful templating language for transforming a JSON dictionary to arbitrary text.

To use this module, you will typically use the Template constructor, and catch various exceptions thrown. You may also want to use the FromFile/FromString methods, which allow Template constructor options to be embedded in the template string itself.

Other functions are exposed for tools which may want to process templates.

exception autotest.client.shared.jsontemplate.Error[source]

Bases: exceptions.Exception

Base class for all exceptions in this module.

Thus you can “except jsontemplate.Error: to catch all exceptions thrown by this module.

exception autotest.client.shared.jsontemplate.CompilationError[source]

Bases: autotest.client.shared.jsontemplate.Error

Base class for errors that happen during the compilation stage.

exception autotest.client.shared.jsontemplate.EvaluationError(msg, original_exception=None)[source]

Bases: autotest.client.shared.jsontemplate.Error

Base class for errors that happen when expanding the template.

This class of errors generally involve the data dictionary or the execution of the formatters.

exception autotest.client.shared.jsontemplate.BadFormatter[source]

Bases: autotest.client.shared.jsontemplate.CompilationError

A bad formatter was specified, e.g. {variable|BAD}

exception autotest.client.shared.jsontemplate.BadPredicate[source]

Bases: autotest.client.shared.jsontemplate.CompilationError

A bad predicate was specified, e.g. {.BAD?}

exception autotest.client.shared.jsontemplate.MissingFormatter[source]

Bases: autotest.client.shared.jsontemplate.CompilationError

Raised when formatters are required, and a variable is missing a formatter.

exception autotest.client.shared.jsontemplate.ConfigurationError[source]

Bases: autotest.client.shared.jsontemplate.CompilationError

Raised when the Template options are invalid and it can’t even be compiled.

exception autotest.client.shared.jsontemplate.TemplateSyntaxError[source]

Bases: autotest.client.shared.jsontemplate.CompilationError

Syntax error in the template text.

exception autotest.client.shared.jsontemplate.UndefinedVariable(msg, original_exception=None)[source]

Bases: autotest.client.shared.jsontemplate.EvaluationError

The template contains a variable not defined by the data dictionary.

autotest.client.shared.jsontemplate.CompileTemplate(template_str, builder=None, meta='{}', format_char='|', more_formatters=<function <lambda>>, more_predicates=<function <lambda>>, default_formatter='str')[source]

Compile the template string, calling methods on the ‘program builder’.

Args:
template_str: The template string. It should not have any compilation
options in the header – those are parsed by FromString/FromFile
builder: The interface of _ProgramBuilder isn’t fixed. Use at your own
risk.

meta: The metacharacters to use, e.g. ‘{}’, ‘[]’.

more_formatters:
Something that can map format strings to formatter functions. One of:
  • A plain dictionary of names -> functions e.g. {‘html’: cgi.escape}
  • A higher-order function which takes format strings and returns formatter functions. Useful for when formatters have parsed arguments.
  • A FunctionRegistry instance for the most control. This allows formatters which takes contexts as well.
more_predicates:
Like more_formatters, but for predicates.
default_formatter: The formatter to use for substitutions that are missing a
formatter. The ‘str’ formatter the “default default” – it just tries to convert the context value to a string in some unspecified manner.
Returns:
The compiled program (obtained from the builder)
Raises:
The various subclasses of CompilationError. For example, if default_formatter=None, and a variable is missing a formatter, then MissingFormatter is raised.

This function is public so it can be used by other tools, e.g. a syntax checking tool run before submitting a template to source control.

autotest.client.shared.jsontemplate.FromString(s, more_formatters=<function <lambda>>, _constructor=None)[source]

Like FromFile, but takes a string.

autotest.client.shared.jsontemplate.FromFile(f, more_formatters=<function <lambda>>, _constructor=None)[source]

Parse a template from a file, using a simple file format.

This is useful when you want to include template options in a data file, rather than in the source code.

The format is similar to HTTP or E-mail headers. The first lines of the file can specify template options, such as the metacharacters to use. One blank line must separate the options from the template body.

Example:

default-formatter: none meta: {{}} format-char: : <blank line required> Template goes here: {{variable:html}}
Args:
f: A file handle to read from. Caller is responsible for opening and closing it.
class autotest.client.shared.jsontemplate.Template(template_str, builder=None, undefined_str=None, **compile_options)[source]

Bases: object

Represents a compiled template.

Like many template systems, the template string is compiled into a program, and then it can be expanded any number of times. For example, in a web app, you can compile the templates once at server startup, and use the expand() method at request handling time. expand() uses the compiled representation.

There are various options for controlling parsing – see CompileTemplate. Don’t go crazy with metacharacters. {}, [], {{}} or <> should cover nearly any circumstance, e.g. generating HTML, CSS XML, JavaScript, C programs, text files, etc.

expand(*args, **kwargs)[source]

Expands the template with the given data dictionary, returning a string.

This is a small wrapper around render(), and is the most convenient interface.

Args:
The JSON data dictionary. Like the builtin dict() constructor, it can take a single dictionary as a positional argument, or arbitrary keyword arguments.
Returns:
The return value could be a str() or unicode() instance, depending on the the type of the template string passed in, and what the types the strings in the dictionary are.
render(data_dict, callback)[source]

Low level method to expands the template piece by piece.

Args:
data_dict: The JSON data dictionary. callback: A callback which should be called with each expanded token.

Example: You can pass ‘f.write’ as the callback to write directly to a file handle.

tokenstream(data_dict)[source]

Yields a list of tokens resulting from expansion.

This may be useful for WSGI apps. NOTE: In the current implementation, the entire expanded template must be stored memory.

NOTE: This is a generator, but JavaScript doesn’t have generators.

autotest.client.shared.jsontemplate.expand(template_str, dictionary, **kwargs)[source]

Free function to expands a template string with a data dictionary.

This is useful for cases where you don’t care about saving the result of compilation (similar to re.match(‘.*’, s) vs DOT_STAR.match(s))

kernel_versions Module

autotest.client.shared.kernel_versions.is_release_candidate(version)[source]
autotest.client.shared.kernel_versions.is_released_kernel(version)[source]
autotest.client.shared.kernel_versions.version_choose_config(version, candidates)[source]
autotest.client.shared.kernel_versions.version_encode(version)[source]
autotest.client.shared.kernel_versions.version_len(version)[source]
autotest.client.shared.kernel_versions.version_limit(version, n)[source]

log Module

autotest.client.shared.log.is_failure(status)[source]
autotest.client.shared.log.is_valid_status(status)[source]
autotest.client.shared.log.log_and_ignore_errors(msg)[source]

A decorator for wrapping functions in a ‘log exception and ignore’ try-except block.

autotest.client.shared.log.record(fn)[source]

Generic method decorator for logging calls under the assumption that return=GOOD, exception=FAIL. The method determines parameters as:

subdir = self.subdir if it exists, or None operation = “class name”.”method name” status = None on GOOD, str(exception) on FAIL

The object using this method must have a job attribute for the logging to actually occur, otherwise the logging will silently fail.

Logging can explicitly be disabled for a call by passing a logged=False parameter

logging_config Module

class autotest.client.shared.logging_config.AllowBelowSeverity(level)[source]

Bases: logging.Filter

Allows only records less severe than a given level (the opposite of what the normal logging level filtering does.

filter(record)[source]

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

class autotest.client.shared.logging_config.LoggingConfig(use_console=True)[source]

Bases: object

add_console_handlers()[source]
add_debug_file_handlers(log_dir, log_name=None)[source]
add_file_handler(file_path, level=10, log_dir=None)[source]
add_stream_handler(stream, level=10)[source]
configure_logging(use_console=True, verbose=False)[source]
console_formatter = <logging.Formatter object>
file_formatter = <logging.Formatter object>
classmethod get_autotest_root()[source]
classmethod get_server_log_dir()[source]
classmethod get_timestamped_log_name(base_name)[source]
global_level = 10
stderr_level = 40
stdout_level = 20
class autotest.client.shared.logging_config.TestingConfig(use_console=True)[source]

Bases: autotest.client.shared.logging_config.LoggingConfig

add_file_handler(*args, **kwargs)[source]
add_stream_handler(*args, **kwargs)[source]
configure_logging(**kwargs)[source]

logging_manager Module

class autotest.client.shared.logging_manager.FdRedirectionLoggingManager[source]

Bases: autotest.client.shared.logging_manager.LoggingManager

A simple extension of LoggingManager to use FdRedirectionStreamManagers, so that managed streams have their underlying FDs redirected.

STREAM_MANAGER_CLASS

alias of _FdRedirectionStreamManager

start_logging()[source]

Begin capturing output to the logging module.

undo_redirect()[source]

Undo the last redirection (that hasn’t yet been undone).

If any subprocesses have been launched since the redirection was performed, they must have ended by the time this is called. Otherwise, this will hang waiting for the logging subprocess to end.

class autotest.client.shared.logging_manager.LoggingFile(prefix='', level=10, logger=<logging.RootLogger object>)[source]

Bases: object

File-like object that will receive messages pass them to the logging infrastructure in an appropriate way.

flush()[source]
isatty()[source]
write(data)[source]

” Writes data only if it constitutes a whole line. If it’s not the case, store it in a buffer and wait until we have a complete line. :param data - Raw data (a string) that will be processed.

writelines(lines)[source]

” Writes itertable of lines

Parameters:lines – An iterable of strings that will be processed.
class autotest.client.shared.logging_manager.LoggingManager[source]

Bases: object

Manages a stack of logging configurations, allowing clients to conveniently add and remove logging destinations. Also keeps a list of StreamManagers to easily direct streams into the logging module.

STREAM_MANAGER_CLASS

alias of _StreamManager

logging_config_object = None
manage_stderr()[source]
manage_stdout()[source]
manage_stream(stream, level, stream_setter)[source]

Tells this manager to manage the given stream. All data written to the stream will be directed to the logging module instead. Must be called before start_logging().

Parameters:
  • stream – stream to manage
  • level – level to log data written to this stream
  • stream_setter – function to set the stream to a new object
redirect(filename)[source]

Redirect output to the specified file

redirect_to_stream(stream)[source]

Redirect output to the given stream

restore()[source]

Same as undo_redirect(). For backwards compatibility with fd_stack.

start_logging()[source]

Begin capturing output to the logging module.

stop_logging()[source]

Restore output to its original state.

tee_redirect(filename, level=None)[source]

Tee output to the specified file

tee_redirect_debug_dir(debug_dir, log_name=None, tag=None)[source]

Tee output to a full new set of debug logs in the given directory.

tee_redirect_to_stream(stream)[source]

Tee output to the given stream

undo_redirect()[source]

Undo the last redirection (that hasn’t yet been undone).

If any subprocesses have been launched since the redirection was performed, they must have ended by the time this is called. Otherwise, this will hang waiting for the logging subprocess to end.

class autotest.client.shared.logging_manager.SortingLoggingFile(prefix='', level_list=[('ERROR', 40), ('WARN', 30), ('INFO', 20), ('DEBUG', 10)], logger=<logging.RootLogger object>)[source]

Bases: autotest.client.shared.logging_manager.LoggingFile

File-like object that will receive messages and pass them to the logging infrastructure. It decides where to pass each line by applying a regex to it and seeing which level it matched.

autotest.client.shared.logging_manager.configure_logging(logging_config, **kwargs)[source]

Configure the logging module using the specific configuration object, which should be an instance of logging_config.LoggingConfig (usually of a subclass). Any keyword args will be passed to the object’s configure_logging() method.

Every entry point should call this method at application startup.

autotest.client.shared.logging_manager.do_not_report_as_logging_caller(func)[source]

Decorator to annotate functions we will tell logging not to log.

autotest.client.shared.logging_manager.get_logging_manager(manage_stdout_and_stderr=False, redirect_fds=False)[source]

Create a LoggingManager that’s managing sys.stdout and sys.stderr.

Every entry point that wants to capture stdout/stderr and/or use LoggingManager to manage a stack of destinations should call this method at application startup.

magic Module

Library used to determine a file MIME type by its magic number, it doesn’t have any external dependencies. Based on work of Jason Petrone (jp_py@jsnp.net), adapted to autotest.

Command Line Usage: Running as ‘python magic.py file_path’ will print a
mime string (or just a description) of the file present on file_path.
API Usage:
magic.guess_type(file_path) - Returns a description of what the file on path ‘file’ contains. This function name was chosen due to a similar function on python standard library ‘mimetypes’.

@license: GPL v2 :copyright: Jason Petrone (jp_py@jsnp.net) 2000 :copyright: Lucas Meneghel Rodrigues (lmr@redhat.com) 2010 @see: http://www.jsnp.net/code/magic.py

class autotest.client.shared.magic.MagicLoggingConfig(use_console=True)[source]

Bases: autotest.client.shared.logging_config.LoggingConfig

configure_logging(results_dir=None, verbose=False)[source]
class autotest.client.shared.magic.MagicTest(offset, t, op, value, msg, mask=None)[source]

Bases: object

Compile a magic database entry so it can be compared with data read from files.

compare(data)[source]

Compare data read from the file with the expected data for this particular mime type register.

Parameters:data – Data read from the file.
test(data)[source]

Compare data read from file with self.value if operator is ‘=’.

Parameters:data – Data read from the file.
Returns:None if no match between data and expected value string. Else, print matching mime type information.
autotest.client.shared.magic.guess_type(filename)[source]

Guess the mimetype of a file based on its filename.

Parameters:filename – File name.
Returns:Mimetype string or description, when appropriate mime not available.

mail Module

Notification email library.

Aims to replace a bunch of different email module wrappers previously used.

class autotest.client.shared.mail.EmailNotificationManager(module='scheduler')[source]

Bases: object

Email notification facility, for use in things like the autotest scheduler.

This facility can use values defined in the autotest settings (global_config.ini) to conveniently send notification emails to the admin of an autotest module.

enqueue_admin(subject, message)[source]

Enqueue an email to the test grid admin.

enqueue_exception_admin(reason)[source]

Enqueue an email containing an exception to the test grid admin.

send(to_string, subject, body)[source]

Send emails to the addresses listed in to_string.

to_string is split into a list which can be delimited by any of:
‘;’, ‘,’, ‘:’ or any whitespace
send_admin(subject, body)[source]

Send an email to this grid admin.

send_queued_admin()[source]

Send all queued emails to the test grid admin.

set_module(module)[source]

Change the name of the module we’re notifying for.

autotest.client.shared.mail.send(from_address, to_addresses, cc_addresses, subject, body, smtp_info, html=None)[source]

Send out an email.

Args:

from_address: The email address to put in the “From:” field. to_addresses: Either a single string or an iterable of

strings to put in the “To:” field of the email.
cc_addresses: Either a single string of an iterable of
strings to put in the “Cc:” field of the email.

subject: The email subject. body: The body of the email. there’s no special

handling of encoding here, so it’s safest to stick to 7-bit ASCII text.

smtp_info: Dictionary with SMTP info. html: Optional HTML content of the message.

mock Module

class autotest.client.shared.mock.Mock(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]

Bases: autotest.client.shared.mock.CallableMixin, autotest.client.shared.mock.NonCallableMock

Create a new Mock object. Mock takes several optional arguments that specify the behaviour of the Mock object:

  • spec: This can be either a list of strings or an existing object (a class or instance) that acts as the specification for the mock object. If you pass in an object then a list of strings is formed by calling dir on the object (excluding unsupported magic attributes and methods). Accessing any attribute not in this list will raise an AttributeError.

    If spec is an object (rather than a list of strings) then mock.__class__ returns the class of the spec object. This allows mocks to pass isinstance tests.

  • spec_set: A stricter variant of spec. If used, attempting to set or get an attribute on the mock that isn’t on the object passed as spec_set will raise an AttributeError.

  • side_effect: A function to be called whenever the Mock is called. See the side_effect attribute. Useful for raising exceptions or dynamically changing return values. The function is called with the same arguments as the mock, and unless it returns DEFAULT, the return value of this function is used as the return value.

    Alternatively side_effect can be an exception class or instance. In this case the exception will be raised when the mock is called.

    If side_effect is an iterable then each call to the mock will return the next value from the iterable. If any of the members of the iterable are exceptions they will be raised instead of returned.

  • return_value: The value returned when the mock is called. By default this is a new Mock (created on first access). See the return_value attribute.

  • wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a Mock object that wraps the corresponding attribute of the wrapped object (so attempting to access an attribute that doesn’t exist will raise an AttributeError).

    If the mock has an explicit return_value set then calls are not passed to the wrapped object and the return_value is returned instead.

  • name: If the mock has a name then it will be used in the repr of the mock. This can be useful for debugging. The name is propagated to child mocks.

Mocks can also be called with arbitrary keyword arguments. These will be used to set attributes on the mock after it is created.

class autotest.client.shared.mock.MagicMock(*args, **kw)[source]

Bases: autotest.client.shared.mock.MagicMixin, autotest.client.shared.mock.Mock

MagicMock is a subclass of Mock with default implementations of most of the magic methods. You can use MagicMock without having to configure the magic methods yourself.

If you use the spec or spec_set arguments then only magic methods that exist in the spec will be created.

Attributes and the return value of a MagicMock will also be MagicMocks.

mock_add_spec(spec, spec_set=False)[source]

Add a spec to a mock. spec can either be an object or a list of strings. Only attributes on the spec can be fetched as attributes from the mock.

If spec_set is True then only attributes on the spec can be set.

autotest.client.shared.mock.patch(target, new=sentinel.DEFAULT, spec=None, create=False, spec_set=None, autospec=None, new_callable=None, **kwargs)[source]

patch acts as a function decorator, class decorator or a context manager. Inside the body of the function or with statement, the target is patched with a new object. When the function/with statement exits the patch is undone.

If new is omitted, then the target is replaced with a MagicMock. If patch is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function. If patch is used as a context manager the created mock is returned by the context manager.

target should be a string in the form ‘package.module.ClassName’. The target is imported and the specified object replaced with the new object, so the target must be importable from the environment you are calling patch from. The target is imported when the decorated function is executed, not at decoration time.

The spec and spec_set keyword arguments are passed to the MagicMock if patch is creating one for you.

In addition you can pass spec=True or spec_set=True, which causes patch to pass in the object being mocked as the spec/spec_set object.

new_callable allows you to specify a different class, or callable object, that will be called to create the new object. By default MagicMock is used.

A more powerful form of spec is autospec. If you set autospec=True then the mock with be created with a spec from the object being replaced. All attributes of the mock will also have the spec of the corresponding attribute of the object being replaced. Methods and functions being mocked will have their arguments checked and will raise a TypeError if they are called with the wrong signature. For mocks replacing a class, their return value (the ‘instance’) will have the same spec as the class.

Instead of autospec=True you can pass autospec=some_object to use an arbitrary object as the spec instead of the one being replaced.

By default patch will fail to replace attributes that don’t exist. If you pass in create=True, and the attribute doesn’t exist, patch will create the attribute for you when the patched function is called, and delete it again afterwards. This is useful for writing tests against attributes that your production code creates at runtime. It is off by by default because it can be dangerous. With it switched on you can write passing tests against APIs that don’t actually exist!

Patch can be used as a TestCase class decorator. It works by decorating each test method in the class. This reduces the boilerplate code when your test methods share a common patchings set. patch finds tests by looking for method names that start with patch.TEST_PREFIX. By default this is test, which matches the way unittest finds tests. You can specify an alternative prefix by setting patch.TEST_PREFIX.

Patch can be used as a context manager, with the with statement. Here the patching applies to the indented block after the with statement. If you use “as” then the patched object will be bound to the name after the “as”; very useful if patch is creating a mock object for you.

patch takes arbitrary keyword arguments. These will be passed to the Mock (or new_callable) on construction.

patch.dict(…), patch.multiple(…) and patch.object(…) are available for alternate use-cases.

autotest.client.shared.mock.call

A tuple for holding the results of a call to a mock, either in the form (args, kwargs) or (name, args, kwargs).

If args or kwargs are empty then a call tuple will compare equal to a tuple without those values. This makes comparisons less verbose:

_Call(('name', (), {})) == ('name',)
_Call(('name', (1,), {})) == ('name', (1,))
_Call(((), {'a': 'b'})) == ({'a': 'b'},)

The _Call object provides a useful shortcut for comparing with call:

_Call(((1, 2), {'a': 3})) == call(1, 2, a=3)
_Call(('foo', (1, 2), {'a': 3})) == call.foo(1, 2, a=3)

If the _Call has no name then it will match any name.

autotest.client.shared.mock.create_autospec(spec, spec_set=False, instance=False, _parent=None, _name=None, **kwargs)[source]

Create a mock object using another object as a spec. Attributes on the mock will use the corresponding attribute on the spec object as their spec.

Functions or methods being mocked will have their arguments checked to check that they are called with the correct signature.

If spec_set is True then attempting to set attributes that don’t exist on the spec object will raise an AttributeError.

If a class is used as a spec then the return value of the mock (the instance of the class) will have the same spec. You can use a class as the spec for an instance object by passing instance=True. The returned mock will only be callable if instances of the mock are callable.

create_autospec also takes arbitrary keyword arguments that are passed to the constructor of the created mock.

class autotest.client.shared.mock.NonCallableMock(spec=None, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]

Bases: autotest.client.shared.mock.Base

A non-callable version of Mock

assert_any_call(*args, **kwargs)[source]

assert the mock has been called with the specified arguments.

The assert passes if the mock has ever been called, unlike assert_called_with and assert_called_once_with that only pass if the call is the most recent one.

assert_called_once_with(*args, **kwargs)[source]

assert that the mock was called exactly once and with the specified arguments.

assert_called_with(*args, **kwargs)[source]

assert that the mock was called with the specified arguments.

Raises an AssertionError if the args and keyword args passed in are different to the last call to the mock.

assert_has_calls(calls, any_order=False)[source]

assert the mock has been called with the specified calls. The mock_calls list is checked for the calls.

If any_order is False (the default) then the calls must be sequential. There can be extra calls before or after the specified calls.

If any_order is True then the calls can be in any order, but they must all appear in mock_calls.

attach_mock(mock, attribute)[source]

Attach a mock as an attribute of this one, replacing its name and parent. Calls to the attached mock will be recorded in the method_calls and mock_calls attributes of this one.

call_args
call_args_list
call_count
called
configure_mock(**kwargs)[source]

Set attributes on the mock through keyword arguments.

Attributes plus return values and side effects can be set on child mocks using standard dot notation and unpacking a dictionary in the method call:

>>> attrs = {'method.return_value': 3, 'other.side_effect': KeyError}
>>> mock.configure_mock(**attrs)
mock_add_spec(spec, spec_set=False)[source]

Add a spec to a mock. spec can either be an object or a list of strings. Only attributes on the spec can be fetched as attributes from the mock.

If spec_set is True then only attributes on the spec can be set.

mock_calls
reset_mock()[source]

Restore the mock object to its initial state.

return_value
side_effect
class autotest.client.shared.mock.NonCallableMagicMock(*args, **kw)[source]

Bases: autotest.client.shared.mock.MagicMixin, autotest.client.shared.mock.NonCallableMock

A version of MagicMock that isn’t callable.

mock_add_spec(spec, spec_set=False)[source]

Add a spec to a mock. spec can either be an object or a list of strings. Only attributes on the spec can be fetched as attributes from the mock.

If spec_set is True then only attributes on the spec can be set.

autotest.client.shared.mock.mock_open(mock=None, read_data='')[source]

A helper function to create a mock to replace the use of open. It works for open called directly or used as a context manager.

The mock argument is the mock object to configure. If None (the default) then a MagicMock will be created for you, with the API limited to methods or attributes available on standard file handles.

read_data is a string for the read method of the file handle to return. This is an empty string by default.

class autotest.client.shared.mock.PropertyMock(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]

Bases: autotest.client.shared.mock.Mock

A mock intended to be used as a property, or other descriptor, on a class. PropertyMock provides __get__ and __set__ methods so you can specify a return value when it is fetched.

Fetching a PropertyMock instance from an object calls the mock, with no args. Setting it calls the mock with the value being set.

openvswitch Module

class autotest.client.shared.openvswitch.OpenVSwitch(tmpdir, db_path=None, db_socket=None, db_pidfile=None, ovs_pidfile=None, dbschema=None, install_prefix=None)[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchSystem

OpenVSwtich class.

clean()[source]

Empty cleanup function

init_db()[source]
init_new()[source]

Create new dbfile without any configuration.

start_ovs_vswitchd()[source]
class autotest.client.shared.openvswitch.OpenVSwitchControl[source]

Bases: object

Class select the best matches control class for installed version of OpenVSwitch.

OpenVSwtich parameters are described in man ovs-vswitchd.conf.db

add_br(br_name)[source]
add_port(br_name, port_name)[source]
add_port_tag(port_name, tag)[source]
add_port_trunk(port_name, trunk)[source]
br_exist(br_name)[source]
check_port_in_br(br_name, port_name)[source]
static convert_version_to_int(version)[source]
Parameters:version – (int) Converted from version string 1.4.0 => int 140
del_br(br_name)[source]
del_port(br_name, port_name)[source]
classmethod get_version()[source]

Get version of installed OpenVSwtich.

Returns:Version of OpenVSwtich.
list_br()[source]
set_vlanmode(port_name, vlan_mode)[source]
status()[source]
class autotest.client.shared.openvswitch.OpenVSwitchControlCli[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchControl, autotest.client.shared.utils.VersionableClass

Class select the best matches control class for installed version of OpenVSwitch.

class autotest.client.shared.openvswitch.OpenVSwitchControlCli_140[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchControlCli, autotest.client.shared.utils.VersionableClass

Don’t use this class directly. This class is automatically selected by OpenVSwitchControl.

add_br(br_name)[source]
add_fake_br(br_name, parent, vlan)[source]
add_port(br_name, port_name)[source]
add_port_tag(port_name, tag)[source]
add_port_trunk(port_name, trunk)[source]
Parameters:trunk – list of vlans id.
br_exist(br_name)[source]
del_br(br_name)[source]
del_port(br_name, port_name)[source]
classmethod is_right_version(version)[source]

Check condition for select control class.

Parameters:version – version of OpenVSwtich
list_br()[source]
list_ports(br_name)[source]
ovs_vsctl(parmas, ignore_status=False)[source]
port_to_br(port_name)[source]

Return bridge which contain port.

Parameters:port_name – Name of port.
Returns:Bridge name or None if there is no bridge which contain port.
set_vlanmode(port_name, vlan_mode)[source]
status()[source]
class autotest.client.shared.openvswitch.OpenVSwitchControlDB[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchControl, autotest.client.shared.utils.VersionableClass

Class select the best matches control class for installed version of OpenVSwitch.

class autotest.client.shared.openvswitch.OpenVSwitchControlDB_140[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchControlDB, autotest.client.shared.utils.VersionableClass

Don’t use this class directly. This class is automatically selected by OpenVSwitchControl.

classmethod is_right_version(version)[source]

Check condition for select control class.

Parameters:version – version of OpenVSwtich
class autotest.client.shared.openvswitch.OpenVSwitchSystem(db_path=None, db_socket=None, db_pidfile=None, ovs_pidfile=None, dbschema=None, install_prefix=None)[source]

Bases: autotest.client.shared.openvswitch.OpenVSwitchControlCli, autotest.client.shared.openvswitch.OpenVSwitchControlDB

OpenVSwtich class.

check()[source]
check_db_daemon()[source]

Check if OVS daemon is started correctly.

check_db_file()[source]

Check if db_file exists.

check_db_socket()[source]

Check if db socket exists.

check_switch_daemon()[source]

Check if OVS daemon is started correctly.

clean()[source]

Empty cleanup function

init_system()[source]

Create new dbfile without any configuration.

is_installed()[source]

Check if OpenVSwitch is already installed in system on default places.

Returns:Version of OpenVSwtich.
class autotest.client.shared.openvswitch.ServiceManager[source]

Bases: autotest.client.shared.openvswitch.ServiceManagerInterface

class autotest.client.shared.openvswitch.ServiceManagerInterface[source]

Bases: autotest.client.shared.utils.VersionableClass

classmethod get_version()[source]

Get version of ServiceManager. :return: Version of ServiceManager.

restart(service_name)[source]
start(service_name)[source]
status(service_name)[source]
stop(service_name)[source]
class autotest.client.shared.openvswitch.ServiceManagerSystemD[source]

Bases: autotest.client.shared.openvswitch.ServiceManagerInterface, autotest.client.shared.utils.VersionableClass

classmethod is_right_version(version)[source]

Check condition for select control class. Function must be re-implemented in new OpenVSwitchControl class. Must be re-implemented for in child class.

Parameters:version – version of OpenVSwtich
restart(service_name)[source]
start(service_name)[source]
status(service_name)[source]
stop(service_name)[source]
class autotest.client.shared.openvswitch.ServiceManagerSysvinit[source]

Bases: autotest.client.shared.openvswitch.ServiceManagerInterface, autotest.client.shared.utils.VersionableClass

classmethod is_right_version(version)[source]

Check condition for select control class. Function must be re-implemented in new OpenVSwitchControl class. Must be re-implemented for in child class.

Parameters:version – version of OpenVSwtich
restart(service_name)[source]
start(service_name)[source]
stop(service_name)[source]

packages Module

class autotest.client.shared.packages.PackageManager(pkgmgr_dir, hostname=None, repo_urls=None, upload_paths=None, do_locking=True, run_function=<function run>, run_function_args=[], run_function_dargs={})[source]

Bases: autotest.client.shared.base_packages.BasePackageManager

pidfile Module

class autotest.client.shared.pidfile.PidFileManager(label, results_dir)[source]

Bases: object

close_file(exit_code, signal_code=0)[source]
open_file()[source]

profiler_manager Module

exception autotest.client.shared.profiler_manager.ProfilerNotPresentError(name, *args, **dargs)[source]

Bases: autotest.client.shared.error.JobError

class autotest.client.shared.profiler_manager.profiler_manager(job)[source]

Bases: object

active()[source]

Returns True if profilers are present and started, False otherwise

add(profiler, *args, **dargs)[source]

Add a profiler

before_start(test)[source]

Override to do any setup needed before actually starting the profilers (this function is called before calling test.before_run_once() and profilers.start() in a profiled run).

current_profilers()[source]

Returns a set of the currently enabled profilers

delete(profiler)[source]

Remove a profiler

load_profiler(profiler, args, dargs)[source]

Given a name and args, loads a profiler, initializes it with the required arguments, and returns an instance of it. Raises a ProfilerNotPresentError if the module isn’t found.

only()[source]

Returns True if job is supposed to be run only with profiling turned on, False otherwise

present()[source]

Indicates if any profilers are enabled

report(test)[source]

Report on all enabled profilers

set_only(value)[source]

Changes the flag which determines whether or not the job is to be run without profilers at all

start(test)[source]

Start all enabled profilers

stop(test)[source]

Stop all enabled profilers

progressbar Module

Basic text progress bar without fancy curses features

class autotest.client.shared.progressbar.ProgressBar(minimum=0, maximum=100, width=77, title='')[source]

Displays interactively the progress of a given task

Inspired/adapted from code.activestate.com recipe #168639

DEFAULT_WIDTH = 77
get_screen_text()[source]

Builds the actual progress bar text

increment(increment, update_screen=True)[source]

Increments the current amount value

update(amount, update_screen=True)[source]

Performs sanity checks and update the current amount

update_screen()[source]

Prints the updated text to the screen

report Module

Module used to parse the autotest job status file and generate a JSON file.

Optionally, we can also generate reports (HTML)

exception autotest.client.shared.report.InvalidAutotestResultDirError(directory)[source]

Bases: exceptions.Exception

exception autotest.client.shared.report.InvalidOutputDirError(directory)[source]

Bases: exceptions.Exception

class autotest.client.shared.report.ReportLoggingConfig(use_console=True)[source]

Bases: autotest.client.shared.logging_config.LoggingConfig

Used with the sole purpose of providing convenient logging setup for this program.

configure_logging(results_dir=None, verbose=False)[source]
class autotest.client.shared.report.ReportOptionParser[source]

Bases: optparse.OptionParser

autotest.client.shared.report.generate_html_report(results_dir, relative_links=True)[source]

Render a job report HTML.

All CSS and javascript are inlined, for more convenience.

Parameters:results_dir – Path to the results directory.
autotest.client.shared.report.generate_json_file(results_dir, relative_links=True)[source]

Generate a JSON file with autotest job summary on a given results directory

Parameters:results_dir – Path to the results directory.
autotest.client.shared.report.get_info_file(filename)[source]

Gets the contents of an autotest info file.

It also and highlights the file contents with possible problems.

Parameters:filename – Info file path.
autotest.client.shared.report.parse_results_dir(results_dir, relative_links=True)[source]

Parse a top level status file and produce a dictionary with job data.

Parameters:dirname – Autotest results directory path
Returns:Dictionary with job data.
autotest.client.shared.report.write_html_report(results_dir, report_path=None, encoding='utf8')[source]

Write an HTML file at report_path, with job data summary.

If no report_path specified, generate one at results_dir/job_report.html.

Parameters:
  • results_dir – Directory with test results.
  • report_path – Path to a report file (optional).
  • encoding – Encoding for output (optional).

service Module

autotest.client.shared.service.ServiceManager(run=<function run>)[source]

Detect which init program is being used, init or systemd and return a class has methods to start/stop services.

# Get the system service manager service_manager = ServiceManager()

# Stating service/unit “sshd” service_manager.start(“sshd”)

# Getting a list of available units units = service_manager.list()

# Disabling and stopping a list of services services_to_disable = [‘ntpd’, ‘httpd’] for s in services_to_disable:

service_manager.disable(s) service_manager.stop(s)
Returns:SysVInitServiceManager or SystemdServiceManager
Return type:_GenericServiceManager
autotest.client.shared.service.SpecificServiceManager(service_name, run=<function run>)[source]

# Get the specific service manager for sshd sshd = SpecificServiceManager(“sshd”) sshd.start() sshd.stop() sshd.reload() sshd.restart() sshd.condrestart() sshd.status() sshd.enable() sshd.disable() sshd.is_enabled()

Parameters:service_name (str) – systemd unit or init.d service to manager
Returns:SpecificServiceManager that has start/stop methods
Return type:_SpecificServiceManager
autotest.client.shared.service.convert_systemd_target_to_runlevel(target)[source]

Convert systemd target to runlevel.

Parameters:target (str) – systemd target
Returns:sys_v runlevel
Return type:str
Raises:ValueError – when systemd target is unknown
autotest.client.shared.service.convert_sysv_runlevel(level)[source]

Convert runlevel to systemd target.

Parameters:level (str or int) – sys_v runlevel
Returns:systemd target
Return type:str
Raises:ValueError – when runlevel is unknown
autotest.client.shared.service.get_name_of_init(run=<function run>)[source]

Determine what executable is PID 1, aka init by checking /proc/1/exe This init detection will only run once and cache the return value.

Returns:executable name for PID 1, aka init
Return type:str
autotest.client.shared.service.sys_v_init_command_generator(command)[source]

Generate lists of command arguments for sys_v style inits.

Parameters:command (str) – start,stop,restart, etc.
Returns:list of commands to pass to utils.run or similar function
Return type:list
autotest.client.shared.service.sys_v_init_result_parser(command)[source]

Parse results from sys_v style commands.

Parameters:command (str.) – command.
Returns:different from the command.

command is status: return true if service is running. command is is_enabled: return true if service is enalbled. command is list: return a dict from service name to status. command is others: return true if operate success.

autotest.client.shared.service.systemd_command_generator(command)[source]

Generate list of command line argument strings for systemctl. One argument per string for compatibility Popen

WARNING: If systemctl detects that it is running on a tty it will use color, pipe to $PAGER, change column sizes and not truncate unit names. Use –no-pager to suppress pager output, or set PAGER=cat in the environment. You may need to take other steps to suppress color output. See https://bugzilla.redhat.com/show_bug.cgi?id=713567

Parameters:command (str) – start,stop,restart, etc.
Returns:list of command and arguments to pass to utils.run or similar functions
Return type:list
autotest.client.shared.service.systemd_result_parser(command)[source]

Parse results from systemd style commands.

Parameters:command (str.) – command.
Returns:different from the command.

command is status: return true if service is running. command is is_enabled: return true if service is enalbled. command is list: return a dict from service name to status. command is others: return true if operate success.

settings Module

A singleton class for accessing global config values.

provides access to global configuration file.

class autotest.client.shared.settings.Settings[source]

Bases: object

check_stand_alone_client_run()[source]
config = None
config_file = '/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/global_config.ini'
get_section_values(sections)[source]

Return a config parser object containing a single section of the global configuration, that can be later written to a file object.

Parameters:section – Tuple with sections we want to turn into a config parser object.
Returns:ConfigParser() object containing all the contents of sections.
get_value(section, key, type=<type 'str'>, default=<object object>, allow_blank=False)[source]
merge_configs(shadow_config)[source]
override_value(section, key, new_value)[source]

Override a value from the config file with a new value.

parse_config_file()[source]
reset_values()[source]

Reset all values to those found in the config files (undoes all overrides).

running_stand_alone_client = False
set_config_files(config_file='/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/global_config.ini', shadow_file='/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/shadow_config.ini')[source]
shadow_file = '/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/shadow_config.ini'
exception autotest.client.shared.settings.SettingsError[source]

Bases: autotest.client.shared.error.AutotestError

exception autotest.client.shared.settings.SettingsValueError[source]

Bases: autotest.client.shared.settings.SettingsError

software_manager Module

Software package management library.

This is an abstraction layer on top of the existing distributions high level package managers. It supports package operations useful for testing purposes, and multiple high level package managers (here called backends). If you want to make this lib to support your particular package manager/distro, please implement the given backend class.

author:Higor Vieira Alves (halves@br.ibm.com)
author:Lucas Meneghel Rodrigues (lmr@redhat.com)
author:Ramon de Carvalho Valle (rcvalle@br.ibm.com)
copyright:IBM 2008-2009
copyright:Red Hat 2009-2010
class autotest.client.shared.software_manager.AptBackend[source]

Bases: autotest.client.shared.software_manager.DpkgBackend

Implements the apt backend for software manager.

Set of operations for the apt package manager, commonly found on Debian and Debian based distributions, such as Ubuntu Linux.

add_repo(repo)[source]

Add an apt repository.

Parameters:repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ maverick universe’
install(name)[source]

Installs package [name].

Parameters:name – Package name.
provides(path)[source]

Return a list of packages that provide [path].

Parameters:path – File path.
remove(name)[source]

Remove package [name].

Parameters:name – Package name.
remove_repo(repo)[source]

Remove an apt repository.

Parameters:repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ maverick universe’
upgrade(name=None)[source]

Upgrade all packages of the system with eventual new versions.

Optionally, upgrade individual packages.

Parameters:name (str) – optional parameter wildcard spec to upgrade
class autotest.client.shared.software_manager.BaseBackend[source]

Bases: object

This class implements all common methods among backends.

install_what_provides(path)[source]

Installs package that provides [path].

Parameters:path – Path to file.
class autotest.client.shared.software_manager.DpkgBackend[source]

Bases: autotest.client.shared.software_manager.BaseBackend

This class implements operations executed with the dpkg package manager.

dpkg is a lower level package manager, used by higher level managers such as apt and aptitude.

INSTALLED_OUTPUT = 'install ok installed'
PACKAGE_TYPE = 'deb'
check_installed(name)[source]
list_all()[source]

List all packages available in the system.

list_files(package)[source]

List files installed by package [package].

Parameters:package – Package name.
Returns:List of paths installed by package.
class autotest.client.shared.software_manager.RpmBackend[source]

Bases: autotest.client.shared.software_manager.BaseBackend

This class implements operations executed with the rpm package manager.

rpm is a lower level package manager, used by higher level managers such as yum and zypper.

PACKAGE_TYPE = 'rpm'
SOFTWARE_COMPONENT_QRY = 'rpm %{NAME} %{VERSION} %{RELEASE} %{SIGMD5} %{ARCH}'
check_installed(name, version=None, arch=None)[source]

Check if package [name] is installed.

Parameters:
  • name – Package name.
  • version – Package version.
  • arch – Package architecture.
list_all(software_components=True)[source]

List all installed packages.

Parameters:software_components – log in a format suitable for the SoftwareComponent schema
list_files(name)[source]

List files installed on the system by package [name].

Parameters:name – Package name.
class autotest.client.shared.software_manager.SoftwareManager[source]

Bases: object

Package management abstraction layer.

It supports a set of common package operations for testing purposes, and it uses the concept of a backend, a helper class that implements the set of operations of a given package management tool.

class autotest.client.shared.software_manager.SoftwareManagerLoggingConfig(use_console=True)[source]

Bases: autotest.client.shared.logging_config.LoggingConfig

Used with the sole purpose of providing logging setup for this program.

configure_logging(results_dir=None, verbose=False)[source]
class autotest.client.shared.software_manager.SystemInspector[source]

Bases: object

System inspector class.

This may grow up to include more complete reports of operating system and machine properties.

get_package_management()[source]

Determine the supported package management systems present on the system. If more than one package management system installed, try to find the best supported system.

class autotest.client.shared.software_manager.YumBackend[source]

Bases: autotest.client.shared.software_manager.RpmBackend

Implements the yum backend for software manager.

Set of operations for the yum package manager, commonly found on Yellow Dog Linux and Red Hat based distributions, such as Fedora and Red Hat Enterprise Linux.

add_repo(url)[source]

Adds package repository located on [url].

Parameters:url – Universal Resource Locator of the repository.
install(name)[source]

Installs package [name]. Handles local installs.

provides(name)[source]

Returns a list of packages that provides a given capability.

Parameters:name – Capability name (eg, ‘foo’).
remove(name)[source]

Removes package [name].

Parameters:name – Package name (eg. ‘ipython’).
remove_repo(url)[source]

Removes package repository located on [url].

Parameters:url – Universal Resource Locator of the repository.
upgrade(name=None)[source]

Upgrade all available packages.

Optionally, upgrade individual packages.

Parameters:name (str) – optional parameter wildcard spec to upgrade
class autotest.client.shared.software_manager.ZypperBackend[source]

Bases: autotest.client.shared.software_manager.RpmBackend

Implements the zypper backend for software manager.

Set of operations for the zypper package manager, found on SUSE Linux.

add_repo(url)[source]

Adds repository [url].

Parameters:url – URL for the package repository.
install(name)[source]

Installs package [name]. Handles local installs.

Parameters:name – Package Name.
provides(name)[source]

Searches for what provides a given file.

Parameters:name – File path.
remove(name)[source]

Removes package [name].

remove_repo(url)[source]

Removes repository [url].

Parameters:url – URL for the package repository.
upgrade(name=None)[source]

Upgrades all packages of the system.

Optionally, upgrade individual packages.

Parameters:name (str) – Optional parameter wildcard spec to upgrade
autotest.client.shared.software_manager.install_distro_packages(distro_pkg_map, interactive=False)[source]

Installs packages for the currently running distribution

This utility function checks if the currently running distro is a key in the distro_pkg_map dictionary, and if there is a list of packages set as its value.

If these conditions match, the packages will be installed using the software manager interface, thus the native packaging system if the currenlty running distro.

Parameters:distro_pkg_map (dict) – mapping of distro name, as returned by utils.get_os_vendor(), to a list of package names
Returns:True if any packages were actually installed, False otherwise

ssh_key Module

syncdata Module

test Module

class autotest.client.shared.test.Subtest[source]

Bases: object

Collect result of subtest of main test.

clean()[source]

Check if cleanup is defined.

For makes test fatal add before implementation of test method decorator @subtest_nocleanup

decored()[source]
failed = 0
classmethod get_full_text_result(format_func=None)[source]
Returns:string with text form of result
classmethod get_result()[source]
Returns:Result of subtests. Format:
tuple(pass/fail,function_name,call_arguments)
classmethod get_text_result(format_func=None)[source]
Returns:string with text form of result
classmethod has_failed()[source]
Returns:If any of subtest not pass return True.
classmethod log_append(msg)[source]

Add log_append to result output.

Parameters:msg – Test of log_append
passed = 0
result = []
static result_to_string(result)[source]

Format of result dict.

result = {
‘result’ : “PASS” / “FAIL”, ‘name’ : class name, ‘args’ : test’s args, ‘kargs’ : test’s kargs, ‘output’ : return of test function,

}

Parameters:result – Result of test.
static result_to_string_debug(result)[source]
Parameters:result – Result of test.
runsubtest(url, *args, **dargs)[source]

Execute another autotest test from inside the current test’s scope.

Parameters:
  • test – Parent test.
  • url – Url of new test.
  • tag – Tag added to test name.
  • args – Args for subtest.
  • dargs – Dictionary with args for subtest.

@iterations: Number of subtest iterations. @profile_only: If true execute one profiled run.

test()[source]

Check if test is defined.

For makes test fatal add before implementation of test method decorator @subtest_fatal

class autotest.client.shared.test.base_test(job, bindir, outputdir)[source]

Bases: object

after_run_once()[source]

Called after every run_once (including from a profiled run when it’s called after stopping the profilers).

analyze_perf_constraints(constraints)[source]
assert_(expr, msg='Assertion failed.')[source]
before_run_once()[source]

Override in tests that need it, will be called before any run_once() call including the profiling run (when it’s called before starting the profilers).

cleanup()[source]
configure_crash_handler()[source]
crash_handler_report()[source]
drop_caches_between_iterations()[source]
execute(iterations=None, test_length=None, profile_only=None, _get_time=<built-in function time>, postprocess_profiled_run=None, constraints=(), *args, **dargs)[source]

This is the basic execute method for the tests inherited from base_test. If you want to implement a benchmark test, it’s better to implement the run_once function, to cope with the profiling infrastructure. For other tests, you can just override the default implementation.

Parameters:
  • test_length – The minimum test length in seconds. We’ll run the run_once function for a number of times large enough to cover the minimum test length.
  • iterations – A number of iterations that we’ll run the run_once function. This parameter is incompatible with test_length and will be silently ignored if you specify both.
  • profile_only – If true run X iterations with profilers enabled. If false run X iterations and one with profiling if profiles are enabled. If None, default to the value of job.default_profile_only.
  • _get_time – [time.time] Used for unit test time injection.
  • postprocess_profiled_run – Run the postprocessing for the profiled run.
initialize()[source]
network_destabilizing = False
postprocess()[source]
postprocess_iteration()[source]
preserve_srcdir = False
process_failed_constraints()[source]
register_after_iteration_hook(iteration_hook)[source]

This is how we expect test writers to register an after_iteration_hook. This adds the method to the list of hooks which are executed after each iteration.

Parameters:iteration_hook – Method to run after each iteration. A valid hook accepts a single argument which is the test object.
register_before_iteration_hook(iteration_hook)[source]

This is how we expect test writers to register a before_iteration_hook. This adds the method to the list of hooks which are executed before each iteration.

Parameters:iteration_hook – Method to run before each iteration. A valid hook accepts a single argument which is the test object.
run_once_profiling(postprocess_profiled_run, *args, **dargs)[source]
setup()[source]
warmup(*args, **dargs)[source]
write_attr_keyval(attr_dict)[source]
write_iteration_keyval(attr_dict, perf_dict, tap_report=None)[source]
write_perf_keyval(perf_dict)[source]
write_test_keyval(attr_dict)[source]
autotest.client.shared.test.runtest(job, url, tag, args, dargs, local_namespace={}, global_namespace={}, before_test_hook=None, after_test_hook=None, before_iteration_hook=None, after_iteration_hook=None)[source]
autotest.client.shared.test.subtest_fatal(function)[source]

Decorator which mark test critical. If subtest fails the whole test ends.

autotest.client.shared.test.subtest_nocleanup(function)[source]

Decorator used to disable cleanup function.

utils Module

Convenience functions for use by tests or whomever.

NOTE: this is a mixin library that pulls in functions from several places Note carefully what the precendece order is

There’s no really good way to do this, as this isn’t a class we can do inheritance with, just a collection of static methods.

class autotest.client.shared.utils.AsyncJob(command, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_level=40, kill_func=None, close_fds=False)[source]

Bases: autotest.client.shared.utils.BgJob

cleanup()[source]
get_stderr()[source]
get_stdout()[source]
output_prepare(stdout_file=None, stderr_file=None)[source]
process_output(stdout=True, final_read=False)[source]

output_prepare must be called prior to calling this

wait_for(timeout=None)[source]

Wait for the process to finish. When timeout is provided, process is safely destroyed after timeout. :param timeout: Acceptable timeout :return: results of this command

class autotest.client.shared.utils.BgJob(command, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_level=40, close_fds=False)[source]

Bases: object

cleanup()[source]
output_prepare(stdout_file=None, stderr_file=None)[source]
process_output(stdout=True, final_read=False)[source]

output_prepare must be called prior to calling this

class autotest.client.shared.utils.CmdResult(command='', stdout='', stderr='', exit_status=None, duration=0)[source]

Bases: object

Command execution result.

command: String containing the command line itself exit_status: Integer exit code of the process stdout: String containing stdout of the process stderr: String containing stderr of the process duration: Elapsed wall clock time running the process

class autotest.client.shared.utils.FileFieldMonitor(status_file, data_to_read, mode_diff, continuously=False, contlogging=False, separator=' +', time_step=0.1)[source]

Bases: object

Monitors the information from the file and reports it’s values.

It gather the information at start and stop of the measurement or continuously during the measurement.

class Monitor(master)[source]

Bases: threading.Thread

Internal monitor class to ensure continuous monitor of monitored file.

run()[source]

Start monitor in thread mode

get_status()[source]
Returns:Status of monitored process average value, time of test and array of monitored values and time step of continuous run.
start()[source]

Start value monitor.

stop()[source]

Stop value monitor.

class autotest.client.shared.utils.ForAll[source]

Bases: list

class autotest.client.shared.utils.ForAllP[source]

Bases: list

Parallel version of ForAll

class autotest.client.shared.utils.ForAllPSE[source]

Bases: list

Parallel version of and suppress exception.

class autotest.client.shared.utils.InterruptedThread(target, args=(), kwargs={})[source]

Bases: threading.Thread

Run a function in a background thread.

join(timeout=None, suppress_exception=False)[source]

Join the thread. If target raised an exception, re-raise it. Otherwise, return the value returned by target.

Parameters:
  • timeout – Timeout value to pass to threading.Thread.join().
  • suppress_exception – If True, don’t re-raise the exception.
run()[source]

Run target (passed to the constructor). No point in calling this function directly. Call start() to make this function run in a new thread.

class autotest.client.shared.utils.Statistic[source]

Bases: object

Class to display and collect average, max and min values of a given data set.

get_average()[source]
get_max()[source]
get_min()[source]
record(value)[source]

Record new value to statistic.

class autotest.client.shared.utils.SystemLoad(pids, advanced=False, time_step=0.1, cpu_cont=False, use_log=False)[source]

Bases: object

Get system and/or process values and return average value of load.

dump(pids=[])[source]

Get the status of monitoring. :param pids: List of PIDs you intend to control. Use pids=[] to control

all defined PIDs.
return:
tuple([cpu load], [memory load]):
([(PID1, (PID1_cpu_meas)), (PID2, (PID2_cpu_meas)), …],

[(PID1, (PID1_mem_meas)), (PID2, (PID2_mem_meas)), …])

PID1_cpu_meas:

average_values[], test_time, cont_meas_values[[]], time_step

PID1_mem_meas:

average_values[], test_time, cont_meas_values[[]], time_step

where average_values[] are the measured values (mem_free,swap,…) which are described in SystemLoad.__init__()-FileFieldMonitor. cont_meas_values[[]] is a list of average_values in the sampling times.

get_cpu_status_string(pids=[])[source]

Convert status to string array. :param pids: List of PIDs you intend to control. Use pids=[] to control

all defined PIDs.
Returns:String format to table.
get_mem_status_string(pids=[])[source]

Convert status to string array. :param pids: List of PIDs you intend to control. Use pids=[] to control

all defined PIDs.
Returns:String format to table.
start(pids=[])[source]

Start monitoring of the process system usage. :param pids: List of PIDs you intend to control. Use pids=[] to control

all defined PIDs.
stop(pids=[])[source]

Stop monitoring of the process system usage. :param pids: List of PIDs you intend to control. Use pids=[] to control

all defined PIDs.
class autotest.client.shared.utils.VersionableClass[source]

Bases: object

VersionableClass provides class hierarchy which automatically select right version of class. Class manipulation is used for this reason. By this reason is: Advantage) Only one version is working in one process. Class is changed in whole process. Disadvantage) Only one version is working in one process.

Example of usage (in utils_unittest):

class FooC(object):
    pass

#Not implemented get_version -> not used for versioning.
class VCP(FooC, VersionableClass):
    def __new__(cls, *args, **kargs):
        VCP.master_class = VCP
        return super(VCP, cls).__new__(cls, *args, **kargs)

    def foo(self):
        pass

class VC2(VCP, VersionableClass):
    @staticmethod
    def get_version():
        return "get_version_from_system"

    @classmethod
    def is_right_version(cls, version):
        if version is not None:
            if "version is satisfied":
                return True
        return False

    def func1(self):
        print "func1"

    def func2(self):
        print "func2"

# get_version could be inherited.
class VC3(VC2, VersionableClass):
    @classmethod
    def is_right_version(cls, version):
        if version is not None:
            if "version+1 is satisfied":
                return True
        return False

    def func2(self):
        print "func2_2"

class M(VCP):
    pass

m = M()   # <- When class is constructed the right version is
          #    automatically selected. In this case VC3 is selected.
m.func2() # call VC3.func2(m)
m.func1() # call VC2.func1(m)
m.foo()   # call VC1.foo(m)

# When controlled "program" version is changed then is necessary call
 check_repair_versions or recreate object.

m.check_repair_versions()

# priority of class. (change place where is method searched first in group
# of verisonable class.)

class PP(VersionableClass):
    def __new__(cls, *args, **kargs):
        PP.master_class = PP
        return super(PP, cls).__new__(cls, *args, **kargs)

class PP2(PP, VersionableClass):
    @staticmethod
    def get_version():
        return "get_version_from_system"

    @classmethod
    def is_right_version(cls, version):
        if version is not None:
            if "version is satisfied":
                return True
        return False

    def func1(self):
        print "PP func1"

class N(VCP, PP):
    pass

n = N()

n.func1() # -> "func2"

n.set_priority_class(PP, [VCP, PP])

n.func1() # -> "PP func1"

Necessary for using: 1) Subclass of versionable class must have implemented class methods get_version and is_right_version. These two methods are necessary for correct version section. Class without this method will be never chosen like suitable class.

2) Every class derived from master_class have to add to class definition inheritance from VersionableClass. Direct inheritance from Versionable Class is use like a mark for manipulation with VersionableClass.

3) Master of VersionableClass have to defined class variable cls.master_class.

classmethod check_repair_versions(master_classes=None)[source]

Check version of versionable class and if version not match repair version to correct version.

Parameters:master_classes (list.) – Check and repair only master_class.
classmethod get_version()[source]

Get version of installed OpenVSwtich. Must be re-implemented for in child class.

Returns:Version or None when get_version is unsuccessful.
classmethod is_right_version(version)[source]

Check condition for select control class. Function must be re-implemented in new OpenVSwitchControl class. Must be re-implemented for in child class.

Parameters:version – version of OpenVSwtich
classmethod set_priority_class(prioritized_class, group_classes)[source]

Set class priority. Limited only for change bases class priority inside one subclass.__bases__ after that continue to another class.

autotest.client.shared.utils.archive_as_tarball(source_dir, dest_dir, tarball_name=None, compression='bz2', verbose=True)[source]

Saves the given source directory to the given destination as a tarball

If the name of the archive is omitted, it will be taken from the source_dir. If it is an absolute path, dest_dir will be ignored. But, if both the destination directory and tarball anem is given, and the latter is not an absolute path, they will be combined.

For archiving directory ‘/tmp’ in ‘/net/server/backup’ as file ‘tmp.tar.bz2’, simply use:

>>> utils.archive_as_tarball('/tmp', '/net/server/backup')

To save the file it with a different name, say ‘host1-tmp.tar.bz2’ and save it under ‘/net/server/backup’, use:

>>> utils.archive_as_tarball('/tmp', '/net/server/backup',
                             'host1-tmp')

To save with gzip compression instead (resulting in the file ‘/net/server/backup/host1-tmp.tar.gz’), use:

>>> utils.archive_as_tarball('/tmp', '/net/server/backup',
                             'host1-tmp', 'gz')
autotest.client.shared.utils.args_to_dict(args)[source]

Convert autoserv extra arguments in the form of key=val or key:val to a dictionary. Each argument key is converted to lowercase dictionary key.

Args:
args - list of autoserv extra arguments.
Returns:
dictionary
autotest.client.shared.utils.ask(question, auto=False)[source]

Raw input with a prompt that emulates logging.

Parameters:
  • question – Question to be asked
  • auto – Whether to return “y” instead of asking the question
autotest.client.shared.utils.aton(sr)[source]

Transform a string to a number(include float and int). If the string is not in the form of number, just return false.

Parameters:sr – string to transfrom
Returns:float, int or False for failed transform
autotest.client.shared.utils.bitlist_to_string(data)[source]

Transform from bit list to ASCII string.

Parameters:data – Bit list to be transformed
autotest.client.shared.utils.close_log_file(filename)[source]
autotest.client.shared.utils.compare_versions(ver1, ver2)[source]

Version number comparison between ver1 and ver2 strings.

>>> compare_tuple("1", "2")
-1
>>> compare_tuple("foo-1.1", "foo-1.2")
-1
>>> compare_tuple("1.2", "1.2a")
-1
>>> compare_tuple("1.2b", "1.2a")
1
>>> compare_tuple("1.3.5.3a", "1.3.5.3b")
-1
Args:
ver1: version string ver2: version string
Returns:
int: 1 if ver1 > ver2
0 if ver1 == ver2

-1 if ver1 < ver2

autotest.client.shared.utils.configure(extra=None, configure='./configure')[source]

Run configure passing in the correct host, build, and target options.

Parameters:
  • extra – extra command line arguments to pass to configure
  • configure – which configure script to use
autotest.client.shared.utils.convert_data_size(size, default_sufix='B')[source]

Convert data size from human readable units to an int of arbitrary size.

Parameters:
  • size – Human readable data size representation (string).
  • default_sufix – Default sufix used to represent data.
Returns:

Int with data size in the appropriate order of magnitude.

autotest.client.shared.utils.convert_ipv4_to_ipv6(ipv4)[source]

Translates a passed in string of an ipv4 address to an ipv6 address.

Parameters:ipv4 – a string of an ipv4 address
autotest.client.shared.utils.cpu_affinity_by_task(pid, vcpu_pid)[source]

This function returns the allowed cpus from the proc entry for each vcpu’s through its task id for a pid(of a VM)

autotest.client.shared.utils.create_subnet_mask(bits)[source]
autotest.client.shared.utils.create_x509_dir(path, cacert_subj, server_subj, passphrase, secure=False, bits=1024, days=1095)[source]

Creates directory with freshly generated: ca-cart.pem, ca-key.pem, server-cert.pem, server-key.pem,

Parameters:
  • path – defines path to directory which will be created
  • cacert_subj – ca-cert.pem subject

:param server_key.csr subject :param passphrase - passphrase to ca-key.pem :param secure = False - defines if the server-key.pem will use a passphrase :param bits = 1024: bit length of keys :param days = 1095: cert expiration

Raises:
autotest.client.shared.utils.delete_pid_file_if_exists(program_name, pid_files_dir=None)[source]

Tries to remove <program_name>.pid from the main autotest directory.

autotest.client.shared.utils.deprecated(func)[source]

This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emitted when the function is used.

autotest.client.shared.utils.display_data_size(size)[source]

Display data size in human readable units.

Parameters:size (int) – Data size, in Bytes.
Returns:Human readable string with data size.
autotest.client.shared.utils.etraceback(prep, exc_info)[source]
Enhanced Traceback formats traceback into lines “prep: line
name: line”
param prep:desired line preposition
param exc_info:sys.exc_info of the exception
return:string which contains beautifully formatted exception
autotest.client.shared.utils.find_command(cmd)[source]

Try to find a command in the PATH, paranoid version.

Parameters:cmd – Command to be found.
Raise:ValueError in case the command was not found.
autotest.client.shared.utils.find_free_port(start_port, end_port, address='localhost')[source]

Return a host free port in the range [start_port, end_port].

Parameters:
  • start_port – First port that will be checked.
  • end_port – Port immediately after the last one that will be checked.
autotest.client.shared.utils.find_free_ports(start_port, end_port, count, address='localhost')[source]

Return count of host free ports in the range [start_port, end_port].

@count: Initial number of ports known to be free in the range. :param start_port: First port that will be checked. :param end_port: Port immediately after the last one that will be checked.

autotest.client.shared.utils.find_substring(string, pattern1, pattern2=None)[source]

Return the match of pattern1 in string. Or return the match of pattern2 if pattern is not matched.

@string: string @pattern1: first pattern want to match in string, must set. @pattern2: second pattern, it will be used if pattern1 not match, optional.

Return: Match substing or None

autotest.client.shared.utils.format_ip_with_mask(ip, mask_bits)[source]
autotest.client.shared.utils.format_str_for_message(msg_str)[source]

Format msg_str so that it can be appended to a message. If msg_str consists of one line, prefix it with a space. If msg_str consists of multiple lines, prefix it with a newline.

Parameters:msg_str – string that will be formatted.
autotest.client.shared.utils.generate_random_id()[source]

Return a random string suitable for use as a qemu id.

autotest.client.shared.utils.generate_random_string(length, ignore_str='!"#$%&\'()*+, -./:;<=>?@[\\]^_`{|}~', convert_str='')[source]

Return a random string using alphanumeric characters.

Parameters:
  • length – Length of the string that will be generated.
  • ignore_str – Characters that will not include in generated string.
  • convert_str – Characters that need to be escaped (prepend “”).
Returns:

The generated random string.

autotest.client.shared.utils.generate_tmp_file_name(file_name, ext=None, directory='/tmp/')[source]

Returns a temporary file name. The file is not created.

autotest.client.shared.utils.get_arch(run_function=<function run>)[source]

Get the hardware architecture of the machine. run_function is used to execute the commands. It defaults to utils.run() but a custom method (if provided) should be of the same schema as utils.run. It should return a CmdResult object and throw a CmdError exception.

autotest.client.shared.utils.get_archive_tarball_name(source_dir, tarball_name, compression)[source]

Get the name for a tarball file, based on source, name and compression

autotest.client.shared.utils.get_children_pids(ppid)[source]

Get all PIDs of children/threads of parent ppid param ppid: parent PID return: list of PIDs of all children/threads of ppid

autotest.client.shared.utils.get_cpu_percentage(function, *args, **dargs)[source]

Returns a tuple containing the CPU% and return value from function call.

This function calculates the usage time by taking the difference of the user and system times both before and after the function call.

autotest.client.shared.utils.get_field(data, param, linestart='', sep=' ')[source]

Parse data from string. :param data: Data to parse.

example:
data:
cpu 324 345 34 5 345 cpu0 34 11 34 34 33 ^^^^ start of line params 0 1 2 3 4
Parameters:
  • param – Position of parameter after linestart marker.
  • linestart – String to which start line with parameters.
  • sep – Separator between parameters regular expression.
autotest.client.shared.utils.get_file(src, dest, permissions=None)[source]

Get a file from src, which can be local or a remote URL

autotest.client.shared.utils.get_full_pci_id(pci_id)[source]

Get full PCI ID of pci_id.

Parameters:pci_id – PCI ID of a device.
autotest.client.shared.utils.get_hash_from_file(hash_path, dvd_basename)[source]

Get the a hash from a given DVD image from a hash file (Hash files are usually named MD5SUM or SHA1SUM and are located inside the download directories of the DVDs)

Parameters:
  • hash_path – Local path to a hash file.
  • cd_image – Basename of a CD image
autotest.client.shared.utils.get_ip_local_port_range()[source]
autotest.client.shared.utils.get_num_logical_cpus_per_socket(run_function=<function run>)[source]

Get the number of cores (including hyperthreading) per cpu. run_function is used to execute the commands. It defaults to utils.run() but a custom method (if provided) should be of the same schema as utils.run. It should return a CmdResult object and throw a CmdError exception.

autotest.client.shared.utils.get_path(base_path, user_path)[source]

Translate a user specified path to a real path. If user_path is relative, append it to base_path. If user_path is absolute, return it as is.

Parameters:
  • base_path – The base path of relative user specified paths.
  • user_path – The user specified path.
autotest.client.shared.utils.get_pid_cpu(pid)[source]

Get the process used cpus.

Parameters:pid – process id
Returns:A list include all cpus the process used
Return type:list
autotest.client.shared.utils.get_pid_from_file(program_name, pid_files_dir=None)[source]

Reads the pid from <program_name>.pid in the autotest directory.

:param program_name the name of the program :return: the pid if the file exists, None otherwise.

autotest.client.shared.utils.get_pid_path(program_name, pid_files_dir=None)[source]
autotest.client.shared.utils.get_process_name(pid)[source]

Get process name from PID. :param pid: PID of process.

autotest.client.shared.utils.get_relative_path(path, reference)[source]

Given 2 absolute paths “path” and “reference”, compute the path of “path” as relative to the directory “reference”.

:param path the absolute path to convert to a relative path :param reference an absolute directory path to which the relative

path will be computed
autotest.client.shared.utils.get_stderr_level(stderr_is_expected)[source]
autotest.client.shared.utils.get_stream_tee_file(stream, level, prefix='')[source]
autotest.client.shared.utils.get_thread_cpu(thread)[source]

Get the light weight process(thread) used cpus.

Parameters:thread (string) – thread checked
Returns:A list include all cpus the thread used
Return type:list
autotest.client.shared.utils.get_unique_name(check, prefix='', suffix='', length=None, skip=None)[source]

Get unique name according to check function, use only 1000 iterations. :param cmp: Function called to discover name uniqueness :param prefix: Name prefix :param suffix: Name suffix :param length: Length of random string, when None use numbers (0,1,2) :param skip: skip n numbers (only when length=None

Raises:StopIteration – In case no unique name obtained in 1000 iterations
Returns:Unique name according to check function
autotest.client.shared.utils.get_unused_port()[source]

Finds a semi-random available port. A race condition is still possible after the port number is returned, if another process happens to bind it.

Returns:
A port number that is unused on both TCP and UDP.
autotest.client.shared.utils.get_vendor_from_pci_id(pci_id)[source]

Check out the device vendor ID according to pci_id.

Parameters:pci_id – PCI ID of a device.
autotest.client.shared.utils.hash(type, input=None)[source]

Returns an hash object of type md5 or sha1. This function is implemented in order to encapsulate hash objects in a way that is compatible with python 2.4 and python 2.6 without warnings.

Note that even though python 2.6 hashlib supports hash types other than md5 and sha1, we are artificially limiting the input values in order to make the function to behave exactly the same among both python implementations.

Parameters:input – Optional input string that will be used to update the hash.
autotest.client.shared.utils.import_site_class(path, module, classname, baseclass, modulefile=None)[source]

Try to import site specific class from site specific file if it exists

Args:

path: full filename of the source file calling this (ie __file__) module: full module name classname: class name to be loaded from site file baseclass: base class object to return when no site file present or

to mixin when site class exists but is not inherited from baseclass

modulefile: module filename

Returns: baseclass if site specific class does not exist, the site specific
class if it exists and is inherited from baseclass or a mixin of the site specific class and baseclass when the site specific class exists and is not inherited from baseclass

Raises: ImportError if the site file exists but imports fails

autotest.client.shared.utils.import_site_function(path, module, funcname, dummy, modulefile=None)[source]

Try to import site specific function from site specific file if it exists

Args:
path: full filename of the source file calling this (ie __file__) module: full module name funcname: function name to be imported from site file dummy: dummy function to return in case there is no function to import modulefile: module filename

Returns: site specific function object or dummy

Raises: ImportError if the site file exists but imports fails

autotest.client.shared.utils.import_site_module(path, module, dummy=None, modulefile=None)[source]

Try to import the site specific module if it exists.

:param path full filename of the source file calling this (ie __file__) :param module full module name :param dummy dummy value to return in case there is no symbol to import :param modulefile module filename

Returns:site specific module or dummy

:raise ImportError if the site file exists but imports fails

autotest.client.shared.utils.import_site_symbol(path, module, name, dummy=None, modulefile=None)[source]

Try to import site specific symbol from site specific file if it exists

:param path full filename of the source file calling this (ie __file__) :param module full module name :param name symbol name to be imported from the site file :param dummy dummy value to return in case there is no symbol to import :param modulefile module filename

Returns:site specific symbol or dummy

:raise ImportError if the site file exists but imports fails

autotest.client.shared.utils.interactive_download(url, output_file, title='', chunk_size=102400)[source]

Interactively downloads a given file url to a given output file

Parameters:
  • url (string) – URL for the file to be download
  • output_file (string) – file name or absolute path on which to save the file to
  • title (string) – optional title to go along the progress bar
  • chunk_size (integer) – amount of data to read at a time
autotest.client.shared.utils.ip_to_long(ip)[source]
autotest.client.shared.utils.is_mounted(src, mount_point, fstype, perm=None, verbose=True, fstype_mtab=None)[source]

Check mount status from /etc/mtab

Parameters:
  • src (string) – mount source
  • mount_point (string) – mount point
  • fstype (string) – file system type
  • perm (string) – mount permission
  • fstype_mtab (str) – file system type in mtab could be different
Returns:

if the src is mounted as expect

Return type:

Boolean

autotest.client.shared.utils.is_port_free(port, address)[source]

Return True if the given port is available for use.

Parameters:port – Port number
autotest.client.shared.utils.is_url(path)[source]

Return true if path looks like a URL

autotest.client.shared.utils.join_bg_jobs(bg_jobs, timeout=None)[source]

Joins the bg_jobs with the current thread.

Returns the same list of bg_jobs objects that was passed in.

autotest.client.shared.utils.kill_process_tree(pid, sig=9)[source]

Signal a process and all of its children.

If the process does not exist – return.

Parameters:
  • pid – The pid of the process to signal.
  • sig – The signal to send to the processes.
autotest.client.shared.utils.lock_file(filename, mode=2)[source]
autotest.client.shared.utils.log_last_traceback(msg=None, log=<function error>)[source]

Writes last traceback into specified log. :param msg: Override the default message. [“Original traceback”] :param log: Where to log the traceback [logging.error]

autotest.client.shared.utils.log_line(filename, line)[source]
Write a line to a file. ‘

‘ is appended to the line.

param filename:Path of file to write to, either absolute or relative to the dir set by set_log_file_dir().
param line:Line to write.
autotest.client.shared.utils.long_to_ip(number)[source]
autotest.client.shared.utils.make(extra='', make='make', timeout=None, ignore_status=False)[source]

Run make, adding MAKEOPTS to the list of options.

Parameters:extra – extra command line arguments to pass to make.
autotest.client.shared.utils.matrix_to_string(matrix, header=None)[source]

Return a pretty, aligned string representation of a nxm matrix.

This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.

Parameters:
  • matrix – Matrix representation (list with n rows of m elements).
  • header – Optional tuple or list with header elements to be displayed.
autotest.client.shared.utils.merge_trees(src, dest)[source]

Merges a source directory tree at ‘src’ into a destination tree at ‘dest’. If a path is a file in both trees than the file in the source tree is APPENDED to the one in the destination tree. If a path is a directory in both trees then the directories are recursively merged with this function. In any other case, the function will skip the paths that cannot be merged (instead of failing).

autotest.client.shared.utils.mount(src, mount_point, fstype, perm=None, verbose=True, fstype_mtab=None)[source]

Mount the src into mount_point of the host.

Src:mount source
Mount_point:mount point
Fstype:file system type
Perm:mount permission
Parameters:fstype_mtab (str) – file system type in mtab could be different
autotest.client.shared.utils.normalize_hostname(alias)[source]
autotest.client.shared.utils.nuke_pid(pid, signal_queue=(15, 9))[source]
autotest.client.shared.utils.nuke_subprocess(subproc)[source]
autotest.client.shared.utils.open_write_close(filename, data)[source]
autotest.client.shared.utils.parallel(targets)[source]

Run multiple functions in parallel.

Parameters:targets – A sequence of tuples or functions. If it’s a sequence of tuples, each tuple will be interpreted as (target, args, kwargs) or (target, args) or (target,) depending on its length. If it’s a sequence of functions, the functions will be called without arguments.
Returns:A list of the values returned by the functions called.
autotest.client.shared.utils.pid_exists(pid)[source]

Return True if a given PID exists.

Parameters:pid – Process ID number.
autotest.client.shared.utils.pid_is_alive(pid)[source]

True if process pid exists and is not yet stuck in Zombie state. Zombies are impossible to move between cgroups, etc. pid can be integer, or text of integer.

autotest.client.shared.utils.process_or_children_is_defunct(ppid)[source]

Verify if any processes from PPID is defunct.

Attempt to verify if parent process and any children from PPID is defunct (zombie) or not. :param ppid: The parent PID of the process to verify.

autotest.client.shared.utils.program_is_alive(program_name, pid_files_dir=None)[source]

Checks if the process is alive and not in Zombie state.

:param program_name the name of the program :return: True if still alive, False otherwise

autotest.client.shared.utils.read_file(filename)[source]
autotest.client.shared.utils.read_keyval(path)[source]

Read a key-value pair format file into a dictionary, and return it. Takes either a filename or directory name as input. If it’s a directory name, we assume you want the file to be called keyval.

autotest.client.shared.utils.read_one_line(filename)[source]
autotest.client.shared.utils.run(command, timeout=None, ignore_status=False, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_is_expected=None, args=())[source]

Run a command on the host.

Parameters:
  • command – the command line string.
  • timeout – time limit in seconds before attempting to kill the running process. The run() function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process.
  • ignore_status – do not raise an exception, no matter what the exit code of the command is.
  • stdout_tee – optional file-like object to which stdout data will be written as it is generated (data will still be stored in result.stdout).
  • stderr_tee – likewise for stderr.
  • verbose – if True, log the command being run.
  • stdin – stdin to pass to the executed process (can be a file descriptor, a file object of a real file or a string).
  • args – sequence of strings of arguments to be given to the command inside ” quotes after they have been escaped for that; each element in the sequence will be given as a separate command argument
Returns:

a CmdResult object

Raises:

CmdError – the exit code of the command execution was not 0

autotest.client.shared.utils.run_bg(*args, **dargs)[source]

Function deprecated. Please use BgJob class instead.

autotest.client.shared.utils.run_parallel(commands, timeout=None, ignore_status=False, stdout_tee=None, stderr_tee=None)[source]

Behaves the same as run() with the following exceptions:

  • commands is a list of commands to run in parallel.
  • ignore_status toggles whether or not an exception should be raised on any error.
Returns:a list of CmdResult objects
class autotest.client.shared.utils.run_randomly(run_sequentially=False)[source]
add(*args, **dargs)[source]
run(fn)[source]
autotest.client.shared.utils.safe_kill(pid, signal)[source]

Attempt to send a signal to a given process that may or may not exist.

Parameters:signal – Signal number.
autotest.client.shared.utils.safe_rmdir(path, timeout=10)[source]

Try to remove a directory safely, even on NFS filesystems.

Sometimes, when running an autotest client test on an NFS filesystem, when not all filedescriptors are closed, NFS will create some temporary files, that will make shutil.rmtree to fail with error 39 (directory not empty). So let’s keep trying for a reasonable amount of time before giving up.

Parameters:
  • path (string) – Path to a directory to be removed.
  • timeout (int) – Time that the function will try to remove the dir before giving up (seconds)
Raises:

OSError, with errno 39 in case after the timeout shutil.rmtree could not successfuly complete. If any attempt to rmtree fails with errno different than 39, that exception will be just raised.

autotest.client.shared.utils.selinux_enforcing()[source]

Returns True if SELinux is in enforcing mode, False if permissive/disabled

autotest.client.shared.utils.set_ip_local_port_range(lower, upper)[source]
autotest.client.shared.utils.set_log_file_dir(directory)[source]

Set the base directory for log files created by log_line().

Parameters:dir – Directory for log files.
autotest.client.shared.utils.sh_escape(command)[source]

Escape special characters from a command so that it can be passed as a double quoted (” “) string in a (ba)sh command.

Args:
command: the command string to escape.
Returns:
The escaped command string. The required englobing double quotes are NOT added and so should be added at some point by the caller.

See also: http://www.tldp.org/LDP/abs/html/escapingsection.html

autotest.client.shared.utils.signal_pid(pid, sig)[source]

Sends a signal to a process id. Returns True if the process terminated successfully, False otherwise.

autotest.client.shared.utils.signal_program(program_name, sig=15, pid_files_dir=None)[source]

Sends a signal to the process listed in <program_name>.pid

:param program_name the name of the program :param sig signal to send

autotest.client.shared.utils.string_to_bitlist(data)[source]

Transform from ASCII string to bit list.

Parameters:data – String to be transformed
autotest.client.shared.utils.strip_console_codes(output)[source]

Remove the Linux console escape and control sequences from the console output. Make the output readable and can be used for result check. Now only remove some basic console codes using during boot up.

Parameters:output (string) – The output from Linux console
Returns:the string wihout any special codes
Return type:string
autotest.client.shared.utils.strip_unicode(input)[source]
autotest.client.shared.utils.system(command, timeout=None, ignore_status=False, verbose=True)[source]

Run a command

Parameters:
  • timeout – timeout in seconds
  • ignore_status – if ignore_status=False, throw an exception if the command’s exit code is non-zero if ignore_status=True, return the exit code.
  • verbose – if True, log the command being run.
Returns:

exit status of command (note, this will always be zero unless ignore_status=True)

autotest.client.shared.utils.system_output(command, timeout=None, ignore_status=False, retain_output=False, args=(), verbose=True)[source]

Run a command and return the stdout output.

Parameters:
  • command – command string to execute.
  • timeout – time limit in seconds before attempting to kill the running process. The function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process.
  • ignore_status – do not raise an exception, no matter what the exit code of the command is.
  • retain_output – set to True to make stdout/stderr of the command output to be also sent to the logging system
  • args – sequence of strings of arguments to be given to the command inside ” quotes after they have been escaped for that; each element in the sequence will be given as a separate command argument
  • verbose – if True, log the command being run.
Returns:

a string with the stdout output of the command.

autotest.client.shared.utils.system_output_parallel(commands, timeout=None, ignore_status=False, retain_output=False)[source]
autotest.client.shared.utils.system_parallel(commands, timeout=None, ignore_status=False)[source]

This function returns a list of exit statuses for the respective list of commands.

autotest.client.shared.utils.umount(src, mount_point, fstype, verbose=True, fstype_mtab=None)[source]

Umount the src mounted in mount_point.

Src:mount source
Mount_point:mount point
Type:file system type
Parameters:fstype_mtab (str) – file system type in mtab could be different
autotest.client.shared.utils.unique(llist)[source]

Return a list of the elements in list, but without duplicates.

Parameters:list – List with values.
Returns:List with non duplicate elements.
autotest.client.shared.utils.unlock_file(lockfile)[source]
autotest.client.shared.utils.unmap_url(srcdir, src, destdir='.')[source]

Receives either a path to a local file or a URL. returns either the path to the local file, or the fetched URL

unmap_url(‘/usr/src’, ‘foo.tar’, ‘/tmp’)
= ‘/usr/src/foo.tar’
unmap_url(‘/usr/src’, ‘http://site/file’, ‘/tmp’)
= ‘/tmp/file’ (after retrieving it)
autotest.client.shared.utils.update_version(srcdir, preserve_srcdir, new_version, install, *args, **dargs)[source]

Make sure srcdir is version new_version

If not, delete it and install() the new version.

In the preserve_srcdir case, we just check it’s up to date, and if not, we rerun install, without removing srcdir

autotest.client.shared.utils.urlopen(url, data=None, timeout=5)[source]

Wrapper to urllib2.urlopen with timeout addition.

autotest.client.shared.utils.urlretrieve(url, filename, data=None, timeout=300)[source]

Retrieve a file from given url.

autotest.client.shared.utils.verify_running_as_root()[source]

Verifies whether we’re running under UID 0 (root).

Raise:error.TestNAError
autotest.client.shared.utils.wait_for(func, timeout, first=0.0, step=1.0, text=None)[source]

If func() evaluates to True before timeout expires, return the value of func(). Otherwise return None.

@brief: Wait until func() evaluates to True.

Parameters:
  • timeout – Timeout in seconds
  • first – Time to sleep before first attempt
  • steps – Time to sleep between attempts in seconds
  • text – Text to print while waiting, for debug purposes
autotest.client.shared.utils.write_keyval(path, dictionary, type_tag=None, tap_report=None)[source]

Write a key-value pair format file out to a file. This uses append mode to open the file, so existing text will not be overwritten or reparsed.

If type_tag is None, then the key must be composed of alphanumeric characters (or dashes+underscores). However, if type-tag is not null then the keys must also have “{type_tag}” as a suffix. At the moment the only valid values of type_tag are “attr” and “perf”.

Parameters:
  • path – full path of the file to be written
  • dictionary – the items to write
  • type_tag – see text above
autotest.client.shared.utils.write_one_line(filename, line)[source]
autotest.client.shared.utils.write_pid(program_name, pid_files_dir=None)[source]

Try to drop <program_name>.pid in the main autotest directory.

Args:
program_name: prefix for file name

utils_cgroup Module

Helpers for cgroup testing.

copyright:2011 Red Hat Inc.
author:Lukas Doktor <ldoktor@redhat.com>
class autotest.client.shared.utils_cgroup.Cgroup(module, _client)[source]

Bases: object

Cgroup handling class.

cgclassify_cgroup(pid, cgroup)[source]

Classify pid into cgroup

Parameters:
  • pid – pid of the process
  • cgroup – cgroup name
cgdelete_all_cgroups()[source]

Delete all cgroups in the module

cgdelete_cgroup(cgroup, recursive=False)[source]

Delete desired cgroup.

Params cgroup:desired cgroup

:params force:If true, sub cgroup can be deleted with parent cgroup

cgexec(cgroup, cmd, args='')[source]

Execute command in desired cgroup

Param:cgroup: Desired cgroup
Param:cmd: Executed command
Param:args: Executed command’s parameters
cgset_property(prop, value, pwd=None, check=True, checkprop=None)[source]

Sets the property value by cgset command

Param:

prop: property name (file)

Param:

value: desired value

Parameters:
  • pwd – cgroup directory
  • check – check the value after setup / override checking value
  • checkprop – override prop when checking the value
get_cgroup_index(cgroup)[source]

Get cgroup’s index in cgroups

Param:cgroup: cgroup name
Returns:index of cgroup
get_cgroup_name(pwd=None)[source]

Get cgroup’s name

Param:pwd: cgroup name
Returns:cgroup’s name
get_pids(pwd=None)[source]

Get all pids in cgroup

Params:pwd: cgroup directory
Returns:all pids(list)
get_property(prop, pwd=None)[source]

Gets the property value :param prop: property name (file) :param pwd: cgroup directory :return: [] values or None when FAILED

initialize(modules)[source]

Initializes object for use.

Parameters:modules – Array of all available cgroup modules.
is_cgroup(pid, pwd)[source]

Checks if the ‘pid’ process is in ‘pwd’ cgroup :param pid: pid of the process :param pwd: cgroup directory :return: 0 when is ‘pwd’ member

is_root_cgroup(pid)[source]

Checks if the ‘pid’ process is in root cgroup (WO cgroup) :param pid: pid of the process :return: 0 when is ‘root’ member

mk_cgroup(pwd=None, cgroup=None)[source]

Creates new temporary cgroup :param pwd: where to create this cgroup (default: self.root) :param cgroup: desired cgroup name :return: last cgroup index

mk_cgroup_cgcreate(pwd=None, cgroup=None)[source]

Make a cgroup by cgcreate command

Params:cgroup: Maked cgroup name
Returns:last cgroup index
refresh_cgroups()[source]

Refresh all cgroups path.

rm_cgroup(pwd)[source]

Removes cgroup.

Parameters:pwd – cgroup directory.
set_cgroup(pid, pwd=None)[source]

Sets cgroup membership :param pid: pid of the process :param pwd: cgroup directory

set_property(prop, value, pwd=None, check=True, checkprop=None)[source]

Sets the property value :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value

set_property_h(prop, value, pwd=None, check=True, checkprop=None)[source]

Sets the one-line property value concerning the K,M,G postfix :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value

set_root_cgroup(pid)[source]

Resets the cgroup membership (sets to root) :param pid: pid of the process :return: 0 when PASSED

smoke_test()[source]

Smoke test Module independent basic tests

test(cmd)[source]

Executes cgroup_client.py with cmd parameter.

Parameters:cmd – command to be executed
Returns:subprocess.Popen() process
class autotest.client.shared.utils_cgroup.CgroupModules(mountdir=None)[source]

Bases: object

Handles the list of different cgroup filesystems.

get_pwd(module)[source]

Returns the mount directory of ‘module’ :param module: desired module (memory, …) :return: mount directory of ‘module’ or None

init(_modules)[source]
Checks the mounted modules and if necessary mounts them into tmp
mountdir.
Parameters:_modules – Desired modules.’memory’,’cpu,cpuset’…
Returns:Number of initialized modules.
autotest.client.shared.utils_cgroup.all_cgroup_delete()[source]

Clear all cgroups in system

autotest.client.shared.utils_cgroup.cgconfig_condrestart()[source]

Condrestart cgconfig service

autotest.client.shared.utils_cgroup.cgconfig_exists()[source]

Check if cgconfig is available on the host or perhaps systemd is used

autotest.client.shared.utils_cgroup.cgconfig_is_running()[source]

Check cgconfig service status

autotest.client.shared.utils_cgroup.cgconfig_restart()[source]

Restart cgconfig service

autotest.client.shared.utils_cgroup.cgconfig_start()[source]

Stop cgconfig service

autotest.client.shared.utils_cgroup.cgconfig_stop()[source]

Start cgconfig service

autotest.client.shared.utils_cgroup.get_all_controllers()[source]

Get all controllers used in system

Returns:all used controllers(controller_list)
autotest.client.shared.utils_cgroup.get_cgroup_mountpoint(controller)[source]

Get desired controller’s mountpoint

@controller: Desired controller :return: controller’s mountpoint

autotest.client.shared.utils_cgroup.get_load_per_cpu(_stats=None)[source]

Gather load per cpu from /proc/stat :param _stats: previous values :return: list of diff/absolute values of CPU times [SUM, CPU1, CPU2, …]

autotest.client.shared.utils_cgroup.resolve_task_cgroup_path(pid, controller)[source]

Resolving cgroup mount path of a particular task

Params:pid : process id of a task for which the cgroup path required
Params:controller: takes one of the controller names in controller list
Returns:resolved path for cgroup controllers of a given pid
autotest.client.shared.utils_cgroup.service_cgconfig_control(action)[source]

Cgconfig control by action.

If cmd executes successfully, return True, otherwise return False. If the action is status, return True when it’s running, otherwise return False. To check if the cgconfig stuff is available, use action “exists”.

@ param action: start|stop|status|restart|condrestart

utils_koji Module

class autotest.client.shared.utils_koji.KojiClient(cmd=None)[source]

Bases: object

Stablishes a connection with the build system, either koji or brew.

This class provides convenience methods to retrieve information on packages and the packages themselves hosted on the build system. Packages should be specified in the KojiPgkSpec syntax.

CMD_LOOKUP_ORDER = ['/usr/bin/brew', '/usr/bin/koji']
CONFIG_MAP = {'/usr/bin/brew': '/etc/brewkoji.conf', '/usr/bin/koji': '/etc/koji.conf'}
get_default_command()[source]

Looks up for koji or brew “binaries” on the system

Systems with plain koji usually don’t have a brew cmd, while systems with koji, have both koji and brew utilities. So we look for brew first, and if found, we consider that the system is configured for brew. If not, we consider this is a system with plain koji.

Returns:either koji or brew command line executable path, or None
get_pkg_base_url()[source]

Gets the base url for packages in Koji

get_pkg_info(pkg)[source]

Returns information from Koji on the package

Parameters:pkg (KojiPkgSpec) – information about the package, as a KojiPkgSpec instance
Returns:information from Koji about the specified package
get_pkg_rpm_file_names(pkg, arch=None)[source]

Gets the file names for the RPM packages specified in pkg

Parameters:
  • pkg (KojiPkgSpec) – a package specification
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_pkg_rpm_info(pkg, arch=None)[source]

Returns a list of information on the RPM packages found on koji

Parameters:
  • pkg (KojiPkgSpec) – a package specification
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_pkg_rpm_names(pkg, arch=None)[source]

Gets the names for the RPM packages specified in pkg

Parameters:
  • pkg (KojiPkgSpec) – a package specification
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_pkg_urls(pkg, arch=None)[source]

Gets the urls for the packages specified in pkg

Parameters:
  • pkg (KojiPkgSpec) – a package specification
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_pkgs(pkg, dst_dir, arch=None)[source]

Download the packages

Parameters:
  • pkg (KojiPkgSpec) – a package specification
  • dst_dir (string) – the destination directory, where the downloaded packages will be saved on
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_scratch_base_url()[source]

Gets the base url for scratch builds in Koji

get_scratch_pkg_urls(pkg, arch=None)[source]

Gets the urls for the scratch packages specified in pkg

Parameters:
  • pkg (KojiScratchPkgSpec) – a scratch package specification
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_scratch_pkgs(pkg, dst_dir, arch=None)[source]

Download the packages from a scratch build

Parameters:
  • pkg (KojiScratchPkgSpec) – a scratch package specification
  • dst_dir (string) – the destination directory, where the downloaded packages will be saved on
  • arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
get_session_options()[source]

Filter only options necessary for setting up a cobbler client session

Returns:only the options used for session setup
is_command_valid()[source]

Checks if the currently set koji command is valid

Returns:True or False
is_config_valid()[source]

Checks if the currently set koji configuration is valid

Returns:True or False
is_pkg_spec_build_valid(pkg)[source]

Checks if build is valid on Koji

Parameters:pkg – a Pkg instance
is_pkg_spec_tag_valid(pkg)[source]

Checks if tag is valid on Koji

Parameters:pkg (KojiPkgSpec) – a package specification
is_pkg_valid(pkg)[source]

Checks if this package is altogether valid on Koji

This verifies if the build or tag specified in the package specification actually exist on the Koji server

Returns:True or False
read_config(check_is_valid=True)[source]

Reads options from the Koji configuration file

By default it checks if the koji configuration is valid

Parameters:check_valid (boolean) – whether to include a check on the configuration
Raise:ValueError
Returns:None
class autotest.client.shared.utils_koji.KojiDirIndexParser[source]

Bases: HTMLParser.HTMLParser

Parser for HTML directory index pages, specialized to look for RPM links

handle_starttag(tag, attrs)[source]

Handle tags during the parsing

This just looks for links (‘a’ tags) for files ending in .rpm

class autotest.client.shared.utils_koji.KojiPkgSpec(text='', tag=None, build=None, package=None, subpackages=[])[source]

Bases: object

A package specification syntax parser for Koji

This holds information on either tag or build, and packages to be fetched from koji and possibly installed (features external do this class).

New objects can be created either by providing information in the textual format or by using the actual parameters for tag, build, package and sub- packages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.

The following sets of examples are interchangeable. Specifying all packages part of build number 1000:

>>> from kvm_utils import KojiPkgSpec
>>> pkg = KojiPkgSpec('1000')
>>> pkg = KojiPkgSpec(build=1000)

Specifying only a subset of packages of build number 1000:

>>> pkg = KojiPkgSpec('1000:kernel,kernel-devel')
>>> pkg = KojiPkgSpec(build=1000,
                      subpackages=['kernel', 'kernel-devel'])

Specifying the latest build for the ‘kernel’ package tagged with ‘dist-f14’:

>>> pkg = KojiPkgSpec('dist-f14:kernel')
>>> pkg = KojiPkgSpec(tag='dist-f14', package='kernel')

Specifying the ‘kernel’ package using the default tag:

>>> kvm_utils.set_default_koji_tag('dist-f14')
>>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')

Specifying the ‘kernel’ package using the default tag:

>>> kvm_utils.set_default_koji_tag('dist-f14')
>>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')

If you do not specify a default tag, and give a package name without an explicit tag, your package specification is considered invalid:

>>> print kvm_utils.get_default_koji_tag()
None
>>> print kvm_utils.KojiPkgSpec('kernel').is_valid()
False
>>> print kvm_utils.KojiPkgSpec(package='kernel').is_valid()
False
SEP = ':'
describe()[source]

Describe this package specification, in a human friendly way

Returns:package specification description
describe_invalid()[source]

Describes why this is not valid, in a human friendly way

is_valid()[source]

Checks if this package specification is valid.

Being valid means that it has enough and not conflicting information. It does not validate that the packages specified actually existe on the Koji server.

Returns:True or False
parse(text)[source]

Parses a textual representation of a package specification

Parameters:text (string) – textual representation of a package in koji
to_text()[source]

Return the textual representation of this package spec

The output should be consumable by parse() and produce the same package specification.

We find that it’s acceptable to put the currently set default tag as the package explicit tag in the textual definition for completeness.

Returns:package specification in a textual representation
class autotest.client.shared.utils_koji.KojiScratchPkgSpec(text='', user=None, task=None, subpackages=[])[source]

Bases: object

A package specification syntax parser for Koji scratch builds

This holds information on user, task and subpackages to be fetched from koji and possibly installed (features external do this class).

New objects can be created either by providing information in the textual format or by using the actual parameters for user, task and subpackages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.

This package definition has a special behaviour: if no subpackages are specified, all packages of the chosen architecture (plus noarch packages) will match.

The following sets of examples are interchangeable. Specifying all packages from a scratch build (whose task id is 1000) sent by user jdoe:

>>> from kvm_utils import KojiScratchPkgSpec
>>> pkg = KojiScratchPkgSpec('jdoe:1000')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000)

Specifying some packages from a scratch build whose task id is 1000, sent by user jdoe:

>>> pkg = KojiScratchPkgSpec('jdoe:1000:kernel,kernel-devel')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000,
                             subpackages=['kernel', 'kernel-devel'])
SEP = ':'
parse(text)[source]

Parses a textual representation of a package specification

Parameters:text (string) – textual representation of a package in koji
class autotest.client.shared.utils_koji.RPMFileNameInfo(filename)[source]

Simple parser for RPM based on information present on the filename itself

get_arch()[source]

Returns just the architecture as present on the RPM filename

get_filename_without_arch()[source]

Returns the filename without the architecture

This also excludes the RPM suffix, that is, removes the leading arch and RPM suffix.

get_filename_without_suffix()[source]

Returns the filename without the default RPM suffix

get_nvr_info()[source]

Returns a dictionary with the name, version and release components

If koji is not installed, this returns None

autotest.client.shared.utils_koji.get_default_koji_tag()[source]
autotest.client.shared.utils_koji.set_default_koji_tag(tag)[source]

Sets the default tag that will be used

utils_memory Module

autotest.client.shared.utils_memory.drop_caches()[source]

Writes back all dirty pages to disk and clears all the caches.

autotest.client.shared.utils_memory.freememtotal()[source]
autotest.client.shared.utils_memory.get_buddy_info(chunk_sizes, nodes='all', zones='all')[source]

Get the fragement status of the host. It use the same method to get the page size in buddyinfo. 2^chunk_size * page_size The chunk_sizes can be string make up by all orders that you want to check splited with blank or a mathematical expression with ‘>’, ‘<’ or ‘=’. For example: The input of chunk_size could be: “0 2 4” And the return will be: {‘0’: 3, ‘2’: 286, ‘4’: 687} if you are using expression: “>=9” the return will be: {‘9’: 63, ‘10’: 225}

Parameters:
  • chunk_size (string) – The order number shows in buddyinfo. This is not the real page size.
  • nodes (string) – The numa node that you want to check. Default value is all
  • zones (string) – The memory zone that you want to check. Default value is all
Returns:

A dict using the chunk_size as the keys

Return type:

dict

autotest.client.shared.utils_memory.get_huge_page_size()[source]
autotest.client.shared.utils_memory.get_num_huge_pages()[source]
autotest.client.shared.utils_memory.memtotal()[source]
autotest.client.shared.utils_memory.node_size()[source]
autotest.client.shared.utils_memory.numa_nodes()[source]
autotest.client.shared.utils_memory.read_from_meminfo(key)[source]
autotest.client.shared.utils_memory.read_from_numa_maps(pid, key)[source]

Get the process numa related info from numa_maps. This function only use to get the numbers like anon=1.

Parameters:
  • pid (String) – Process id
  • key (String) – The item you want to check from numa_maps
Returns:

A dict using the address as the keys

Return type:

dict

autotest.client.shared.utils_memory.read_from_smaps(pid, key)[source]

Get specific item value from the smaps of a process include all sections.

Parameters:
  • pid (String) – Process id
  • key (String) – The item you want to check from smaps
Returns:

The value of the item in kb

Return type:

int

autotest.client.shared.utils_memory.read_from_vmstat(key)[source]

Get specific item value from vmstat

Parameters:key (String) – The item you want to check from vmstat
Returns:The value of the item
Return type:int
autotest.client.shared.utils_memory.rounded_memtotal()[source]
autotest.client.shared.utils_memory.set_num_huge_pages(num)[source]

version Module

Based on work from Douglas Creager <dcreager@dcreager.net>

Gets the current version number. If possible, this is the output of “git describe”, modified to conform to the versioning scheme that setuptools uses. If “git describe” returns an error (most likely because we’re in an unpacked copy of a release tarball, rather than in a git working copy), then we fall back on reading the contents of the RELEASE-VERSION file.

To use this script, simply import it your setup.py file, and use the results of get_version() as your package version:

from autotest.client.shared import version

setup(
version=get_version(), . . .

)

This will automatically update the RELEASE-VERSION file, if necessary. Note that the RELEASE-VERSION file should not be checked into git; please add it to your top-level .gitignore file.

You’ll probably want to distribute the RELEASE-VERSION file in your sdist tarballs; to do this, just create a MANIFEST.in file that contains the following line:

include RELEASE-VERSION

autotest.client.shared.version.get_version(abbrev=4)[source]