shared Package¶
autotemp
Module¶
Autotest tempfile wrapper for mkstemp (known as tempfile here) and mkdtemp (known as tempdir).
This wrapper provides a mechanism to clean up temporary files/dirs once they are no longer need.
Files/Dirs will have a unique_id prepended to the suffix and a _autotmp_ tag appended to the prefix.
It is required that the unique_id param is supplied when a temp dir/file is created.
-
class
autotest.client.shared.autotemp.
tempdir
(suffix='', unique_id=None, prefix='', dir=None)[source]¶ Bases:
object
A wrapper for tempfile.mkdtemp
@var name: The name of the temporary dir. :return: A tempdir object example usage:
b = autotemp.tempdir(unique_id=’exemdir’) b.name # your directory b.clean() # clean up after yourself
-
class
autotest.client.shared.autotemp.
tempfile
(unique_id, suffix='', prefix='', dir=None, text=False)[source]¶ Bases:
object
A wrapper for tempfile.mkstemp
Parameters: unique_id – required, a unique string to help identify what part of code created the tempfile. @var name: The name of the temporary file. @var fd: the file descriptor of the temporary file that was created. :return: a tempfile object example usage:
t = autotemp.tempfile(unique_id=’fig’) t.name # name of file t.fd # file descriptor t.fo # file object t.clean() # clean up after yourself
barrier
Module¶
base_barrier
Module¶
-
exception
autotest.client.shared.base_barrier.
BarrierAbortError
[source]¶ Bases:
autotest.client.shared.error.BarrierError
Special BarrierError raised when an explicit abort is requested.
-
class
autotest.client.shared.base_barrier.
barrier
(hostid, tag, timeout=None, port=None, listen_server=None)[source]¶ Bases:
object
Multi-machine barrier support.
Provides multi-machine barrier mechanism. Execution stops until all members arrive at the barrier.
Implementation Details:
When a barrier is forming the master node (first in sort order) in the set accepts connections from each member of the set. As they arrive they indicate the barrier they are joining and their identifier (their hostname or IP address and optional tag). They are then asked to wait. When all members are present the master node then checks that each member is still responding via a ping/pong exchange. If this is successful then everyone has checked in at the barrier. We then tell everyone they may continue via a rlse message.
Where the master is not the first to reach the barrier the client connects will fail. Client will retry until they either succeed in connecting to master or the overall timeout is exceeded.
As an example here is the exchange for a three node barrier called ‘TAG’
- MASTER CLIENT1 CLIENT2
<————-TAG C1————- ————–wait————–>
[…]<————-TAG C2—————————– ————–wait——————————>
[…]————–ping————–> <————-pong————— ————–ping——————————> <————-pong——————————-
—– BARRIER conditions MET —–————–rlse————–> ————–rlse——————————>
Note that once the last client has responded to pong the barrier is implicitly deemed satisifed, they have all acknowledged their presence. If we fail to send any of the rlse messages the barrier is still a success, the failed host has effectively broken ‘right at the beginning’ of the post barrier execution window.
In addition, there is another rendezvous, that makes each slave a server and the master a client. The connection process and usage is still the same but allows barriers from machines that only have a one-way connection initiation. This is called rendezvous_servers.
- For example:
- if ME == SERVER:
- server start
b = job.barrier(ME, ‘server-up’, 120) b.rendezvous(CLIENT, SERVER)
- if ME == CLIENT:
- client run
b = job.barrier(ME, ‘test-complete’, 3600) b.rendezvous(CLIENT, SERVER)
- if ME == SERVER:
- server stop
Any client can also request an abort of the job by setting abort=True in the rendezvous arguments.
-
class
autotest.client.shared.base_barrier.
listen_server
(address='', port=11922)[source]¶ Bases:
object
Manages a listening socket for barrier.
Can be used to run multiple barrier instances with the same listening socket (if they were going to listen on the same port).
Attributes:
Attr address: Address to bind to (string). Attr port: Port to bind to. Attr socket: Listening socket object.
base_check_version
Module¶
base_job
Module¶
-
class
autotest.client.shared.base_job.
TAPReport
(enable, resultdir=None, global_filename='status')[source]¶ Bases:
object
Deal with TAP reporting for the Autotest client.
-
job_statuses
= {'ABORT': False, 'ALERT': False, 'END GOOD': True, 'ERROR': False, 'FAIL': False, 'GOOD': True, 'NOSTATUS': False, 'RUNNING': False, 'START': True, 'TEST_NA': False, 'WARN': False}¶
-
record
(log_entry, indent, log_files)[source]¶ Append a job-level status event to self._reports_container. All events will be written to TAP log files at the end of the test run. Otherwise, it’s impossilble to determine the TAP plan.
Parameters: - log_entry – A string status code describing the type of status entry being recorded. It must pass log.is_valid_status to be considered valid.
- indent – Level of the log_entry to determine the operation if log_entry.operation is not given.
- log_files – List of full path of files the TAP report will be written to at the end of the test.
-
record_keyval
(path, dictionary, type_tag=None)[source]¶ Append a key-value pairs of dictionary to self._keyval_container in TAP format. Once finished write out the keyval.tap file to the file system.
If type_tag is None, then the key must be composed of alphanumeric characters (or dashes + underscores). However, if type-tag is not null then the keys must also have “{type_tag}” as a suffix. At the moment the only valid values of type_tag are “attr” and “perf”.
Parameters: - path – The full path of the keyval.tap file to be created
- dictionary – The keys and values.
- type_tag – The type of the values
-
-
class
autotest.client.shared.base_job.
base_job
(*args, **dargs)[source]¶ Bases:
object
An abstract base class for the various autotest job classes.
Property autodir: The top level autotest directory. Property clientdir: The autotest client directory. Property serverdir: The autotest server directory. [OPTIONAL] Property resultdir: The directory where results should be written out. [WRITABLE] Property pkgdir: The job packages directory. [WRITABLE] Property tmpdir: The job temporary directory. [WRITABLE] Property testdir: The job test directory. [WRITABLE] Property customtestdir: The custom test directory. [WRITABLE] Property site_testdir: The job site test directory. [WRITABLE] Property bindir: The client bin/ directory. Property configdir: The client config/ directory. Property profdir: The client profilers/ directory. Property toolsdir: The client tools/ directory. Property conmuxdir: The conmux directory. [OPTIONAL] Property control: A path to the control file to be executed. [OPTIONAL] Property hosts: A set of all live Host objects currently in use by the job. Code running in the context of a local client can safely assume that this set contains only a single entry. Property machines: A list of the machine names associated with the job. Property user: The user executing the job. Property tag: A tag identifying the job. Often used by the scheduler to give a name of the form NUMBER-USERNAME/HOSTNAME. Property args: A list of additional miscellaneous command-line arguments provided when starting the job. Property last_boot_tag: The label of the kernel from the last reboot. [OPTIONAL,PERSISTENT] Property automatic_test_tag: A string which, if set, will be automatically added to the test name when running tests. Property default_profile_only: A boolean indicating the default value of profile_only used by test.execute. [PERSISTENT] Property drop_caches: A boolean indicating if caches should be dropped before each test is executed. Property drop_caches_between_iterations: A boolean indicating if caches should be dropped before each test iteration is executed. Property run_test_cleanup: A boolean indicating if test.cleanup should be run by default after a test completes, if the run_cleanup argument is not specified. [PERSISTENT] Property num_tests_run: The number of tests run during the job. [OPTIONAL] Property num_tests_failed: The number of tests failed during the job. [OPTIONAL] Property bootloader: An instance of the boottool class. May not be available on job instances where access to the bootloader is not available (e.g. on the server running a server job). [OPTIONAL] Property harness: An instance of the client test harness. Only available in contexts where client test execution happens. [OPTIONAL] Property logging: An instance of the logging manager associated with the job. Property profilers: An instance of the profiler manager associated with the job. Property sysinfo: An instance of the sysinfo object. Only available in contexts where it’s possible to collect sysinfo. Property warning_manager: A class for managing which types of WARN messages should be logged and which should be suppressed. [OPTIONAL] Property warning_loggers: A set of readable streams that will be monitored for WARN messages to be logged. [OPTIONAL] - Abstract methods:
- _find_base_directories [CLASSMETHOD]
- Returns the location of autodir, clientdir and serverdir
- _find_resultdir
- Returns the location of resultdir. Gets a copy of any parameters passed into base_job.__init__. Can return None to indicate that no resultdir is to be used.
- _get_status_logger
- Returns a status_logger instance for recording job status logs.
-
autodir
¶
-
automatic_test_tag
¶
-
bindir
¶
-
clientdir
¶
-
configdir
¶
-
conmuxdir
¶
-
customtestdir
¶
-
default_profile_only
¶
-
get_state
(name, default=<object object>)[source]¶ Returns the value associated with a particular name.
Parameters: - name – The name the value was saved with.
- default – A default value to return if no state is currently associated with var.
Returns: A deep copy of the value associated with name. Note that this explicitly returns a deep copy to avoid problems with mutable values; mutations are not persisted or shared.
Raises: KeyError
when no state is associated with var and a default value is not provided.
-
last_boot_tag
¶
-
pkgdir
¶
-
pop_execution_context
()[source]¶ Reverse the effects of the previous push_execution_context call.
Raises: IndexError
when the stack of contexts is empty.
-
profdir
¶
-
push_execution_context
(resultdir)[source]¶ Save off the current context of the job and change to the given one.
In practice method just changes the resultdir, but it may become more extensive in the future. The expected use case is for when a child job needs to be executed in some sort of nested context (for example the way parallel_simple does). The original context can be restored with a pop_execution_context call.
Parameters: resultdir – The new resultdir, relative to the current one.
-
record
(status_code, subdir, operation, status='', optional_fields=None)[source]¶ Record a job-level status event.
Logs an event noteworthy to the Autotest job as a whole. Messages will be written into a global status log file, as well as a subdir-local status log file (if subdir is specified).
Parameters: - status_code – A string status code describing the type of status entry being recorded. It must pass log.is_valid_status to be considered valid.
- subdir – A specific results subdirectory this also applies to, or None. If not None the subdirectory must exist.
- operation – A string describing the operation that was run.
- status – An optional human-readable message describing the status entry, for example an error message or “completed successfully”.
- optional_fields – An optional dictionary of additional named fields to be included with the status message. Every time timestamp and localtime entries are generated with the current time and added to this dictionary.
-
record_entry
(entry, log_in_subdir=True)[source]¶ Record a job-level status event, using a status_log_entry.
This is the same as self.record but using an existing status log entry object rather than constructing one for you.
Parameters: - entry – A status_log_entry object
- log_in_subdir – A boolean that indicates (when true) that subdir logs should be written into the subdirectory status log file.
-
resultdir
¶
-
run_test_cleanup
¶
-
serverdir
¶
-
set_state
(name, value)[source]¶ Saves the value given with the provided name.
Parameters: - name – The name the value should be saved with.
- value – The value to save.
-
site_testdir
¶
-
tag
¶
-
testdir
¶
-
tmpdir
¶
-
toolsdir
¶
-
use_sequence_number
¶
-
class
autotest.client.shared.base_job.
job_directory
(path, is_writable=False)[source]¶ Bases:
object
Represents a job.*dir directory.
-
exception
JobDirectoryException
[source]¶ Bases:
autotest.client.shared.error.AutotestError
Generic job_directory exception superclass.
-
exception
MissingDirectoryException
(path)[source]¶ Bases:
autotest.client.shared.base_job.JobDirectoryException
Raised when a directory required by the job does not exist.
-
exception
UncreatableDirectoryException
(path, error)[source]¶ Bases:
autotest.client.shared.base_job.JobDirectoryException
Raised when a directory required by the job is missing and cannot be created.
-
exception
UnwritableDirectoryException
(path)[source]¶ Bases:
autotest.client.shared.base_job.JobDirectoryException
Raised when a writable directory required by the job exists but is not writable.
-
static
property_factory
(attribute)[source]¶ Create a job.*dir -> job._*dir.path property accessor.
Parameters: attribute – A string with the name of the attribute this is exposed as. ‘_’+attribute must then be attribute that holds either None or a job_directory-like object Returns: A read-only property object that exposes a job_directory path
-
exception
-
class
autotest.client.shared.base_job.
job_state
[source]¶ Bases:
object
A class for managing explicit job and user state, optionally persistent.
The class allows you to save state by name (like a dictionary). Any state stored in this class should be picklable and deep copyable. While this is not enforced it is recommended that only valid python identifiers be used as names. Additionally, the namespace ‘stateful_property’ is used for storing the valued associated with properties constructed using the property_factory method.
-
NO_DEFAULT
= <object object>¶
-
PICKLE_PROTOCOL
= 2¶
-
discard
(*args, **dargs)[source]¶ If namespace.name is a defined value, deletes it.
Parameters: - namespace (string) – The namespace that the property should be stored in.
- name (string) – The name the value was saved with.
-
discard_namespace
(*args, **dargs)[source]¶ Delete all defined namespace.* names.
Parameters: namespace (string) – The namespace to be cleared.
-
get
(*args, **dargs)[source]¶ Returns the value associated with a particular name.
Parameters: - namespace (string) – The namespace that the property should be stored in.
- name (string) – The name the value was saved with.
- default (object) – A default value to return if no state is currently associated with var.
Returns: A deep copy of the value associated with name. Note that this explicitly returns a deep copy to avoid problems with mutable values; mutations are not persisted or shared.
Raises: KeyError
raised when no state is associated with var and a default value is not provided.
-
has
(*args, **dargs)[source]¶ Return a boolean indicating if namespace.name is defined.
Parameters: - namespace (string) – The namespace that the property should be stored in.
- name (string) – The name the value was saved with.
Returns: True if the given name is defined in the given namespace and False otherwise.
Return type:
-
static
property_factory
(state_attribute, property_attribute, default, namespace='global_properties')[source]¶ Create a property object for an attribute using self.get and self.set.
Parameters: - state_attribute – A string with the name of the attribute on job that contains the job_state instance.
- property_attribute – A string with the name of the attribute this property is exposed as.
- default – A default value that should be used for this property if it is not set.
- namespace – The namespace to store the attribute value in.
Returns: A read-write property object that performs self.get calls to read the value and self.set calls to set it.
-
read_from_file
(file_path, merge=True)[source]¶ Read in any state from the file at file_path.
When
merge=True
, any state specified only in-memory will be preserved. Any state specified on-disk will be set in-memory, even if an in-memory setting already exists.Parameters: - file_path (string) – The path where the state should be read from. It must exist but it can be empty.
- merge (bool) – If true, merge the on-disk state with the in-memory state. If false, replace the in-memory state with the on-disk state.
Warning: This method is intentionally concurrency-unsafe. It makes no attempt to control concurrent access to the file at
file_path
.
-
set
(*args, **dargs)[source]¶ Saves the value given with the provided name.
Parameters: - namespace (string) – The namespace that the property should be stored in.
- name (string) – The name the value was saved with.
- value – The value to save.
-
set_backing_file
(file_path)[source]¶ Change the path used as the backing file for the persistent state.
When a new backing file is specified if a file already exists then its contents will be added into the current state, with conflicts between the file and memory being resolved in favor of the file contents. The file will then be kept in sync with the (combined) in-memory state. The syncing can be disabled by setting this to None.
Parameters: file_path (string) – A path on the filesystem that can be read from and written to, or None to turn off the backing store.
-
write_to_file
(file_path)[source]¶ Write out the current state to the given path.
Warning: This method is intentionally concurrency-unsafe. It makes no attempt to control concurrent access to the file at file_path.
Parameters: file_path (string) – The path where the state should be written out to. Must be writable.
-
-
class
autotest.client.shared.base_job.
status_indenter
[source]¶ Bases:
object
Abstract interface that a status log indenter should use.
-
indent
¶
-
-
class
autotest.client.shared.base_job.
status_log_entry
(status_code, subdir, operation, message, fields, timestamp=None)[source]¶ Bases:
object
Represents a single status log entry.
-
BAD_CHAR_REGEX
= <_sre.SRE_Pattern object>¶
-
LOCALTIME_FIELD
= 'localtime'¶
-
RENDERED_NONE_VALUE
= '----'¶
-
TIMESTAMP_FIELD
= 'timestamp'¶
-
is_end
()[source]¶ Indicates if this status log is the end of a nested block.
Returns: A boolean indicating if this entry ends a nested block.
-
is_start
()[source]¶ Indicates if this status log is the start of a new nested block.
Returns: A boolean indicating if this entry starts a new nested block.
-
classmethod
parse
(line)[source]¶ Parse a status log entry from a text string.
This method is the inverse of render; it should always be true that parse(entry.render()) produces a new status_log_entry equivalent to entry.
Returns: A new status_log_entry instance with fields extracted from the given status line. If the line is an extra message line then None is returned.
-
-
class
autotest.client.shared.base_job.
status_logger
(job, indenter, global_filename='status', subdir_filename='status', record_hook=None, tap_writer=None)[source]¶ Bases:
object
Represents a status log file. Responsible for translating messages into on-disk status log lines.
Property global_filename: The filename to write top-level logs to. Property subdir_filename: The filename to write subdir-level logs to. -
record_entry
(log_entry, log_in_subdir=True)[source]¶ Record a status_log_entry into the appropriate status log files.
Parameters: - log_entry – A status_log_entry instance to be recorded into the status logs.
- log_in_subdir – A boolean that indicates (when true) that subdir logs should be written into the subdirectory status log file.
-
-
autotest.client.shared.base_job.
with_backing_file
(method)[source]¶ A decorator to perform a lock-read-*-write-unlock cycle.
When applied to a method, this decorator will automatically wrap calls to the method in a lock-and-read before the call followed by a write-and-unlock. Any operation that is reading or writing state should be decorated with this method to ensure that backing file state is consistently maintained.
base_packages
Module¶
This module defines the BasePackageManager Class which provides an implementation of the packaging system API providing methods to fetch, upload and remove packages. Site specific extensions to any of these methods should inherit this class.
-
class
autotest.client.shared.base_packages.
BasePackageManager
(pkgmgr_dir, hostname=None, repo_urls=None, upload_paths=None, do_locking=True, run_function=<function run>, run_function_args=[], run_function_dargs={})[source]¶ Bases:
object
-
compare_checksum
(pkg_path, repo_url)[source]¶ Calculate the checksum of the file specified in pkg_path and compare it with the checksum in the checksum file Return True if both match else return False. :param pkg_path: The full path to the package file for which the checksum is being compared :param repo_url: The URL to fetch the checksum from
-
compute_checksum
(pkg_path)[source]¶ Compute the MD5 checksum for the package file and return it. pkg_path : The complete path for the package file
-
fetch_pkg
(pkg_name, dest_path, repo_url=None, use_checksum=False, install=False)[source]¶ Fetch the package into dest_dir from repo_url. By default repo_url is None and the package is looked in all the repositories specified. Otherwise it fetches it from the specific repo_url. pkg_name : name of the package (ex: test-sleeptest.tar.bz2,
dep-gcc.tar.bz2, kernel.1-1.rpm)repo_url : the URL of the repository where the package is located. dest_path : complete path of where the package will be fetched to. use_checksum : This is set to False to fetch the packages.checksum file
so that the checksum comparison is bypassed for the checksum file itself. This is used internally by the packaging system. It should be ignored by externals callers of this method who use it fetch custom packages.- install : install path has unique name and destination requirements
- that vary based on the fetcher that is used. So call them here as opposed to install_pkg.
-
get_mirror_list
(repo_urls)[source]¶ Stub function for site specific mirrors.
- Returns:
- Priority ordered list
-
get_package_name
(url, pkg_type)[source]¶ Extract the group and test name for the url. This method is currently used only for tests.
-
static
get_tarball_name
(name, pkg_type)[source]¶ Converts a package name and type into a tarball name.
Parameters: - name – The name of the package
- pkg_type – The type of the package
Returns: A tarball filename for that specific type of package
-
install_pkg
(name, pkg_type, fetch_dir, install_dir, preserve_install_dir=False, repo_url=None)[source]¶ Remove install_dir if it already exists and then recreate it unless preserve_install_dir is specified as True. Fetch the package into the pkg_dir. Untar the package into install_dir The assumption is that packages are of the form : <pkg_type>.<pkg_name>.tar.bz2 name : name of the package type : type of the package fetch_dir : The directory into which the package tarball will be
fetched to.install_dir : the directory where the package files will be untarred to repo_url : the url of the repository to fetch the package from.
-
static
parse_tarball_name
(tarball_name)[source]¶ Coverts a package tarball name into a package name and type.
Parameters: tarball_name – The filename of the tarball Returns: (name, pkg_type) where name is the package name and pkg_type is the package type.
-
remove_checksum
(pkg_name)[source]¶ Remove the checksum of the package from the packages checksum file. This method is called whenever a package is removed from the repositories in order clean its corresponding checksum. pkg_name : The name of the package to be removed
-
remove_pkg
(pkg_name, remove_path=None, remove_checksum=False)[source]¶ Remove the package from the specified remove_path pkg_name : name of the package (ex: test-sleeptest.tar.bz2,
dep-gcc.tar.bz2)remove_path : the location to remove the package from.
-
repo_check
(repo)[source]¶ Check to make sure the repo is in a sane state: ensure we have at least XX amount of free space Make sure we can write to the repo
-
tar_package
(pkg_name, src_dir, dest_dir, include_string=None, exclude_string=None)[source]¶ Create a tar.bz2 file with the name ‘pkg_name’ say test-blah.tar.bz2.
Includes the files specified in include_string, and excludes the files specified on the exclude string, while tarring the source. Returns the destination tarball path.
Parameters: - pkg_name – Package name.
- src_dir – Directory that contains the data to be packaged.
- dest_dir – Directory that will hold the destination tarball.
- include_string – Pattern that represents the files that will be added to the tar package.
- exclude_string – Pattern that represents the files that should be excluded from the tar package. It could be either a string or a list.
-
untar_pkg
(tarball_path, dest_dir)[source]¶ Untar the package present in the tarball_path and put a “.checksum” file in the dest_dir containing the checksum of the tarball. This method assumes that the package to be untarred is of the form <name>.tar.bz2
-
untar_required
(tarball_path, dest_dir)[source]¶ Compare the checksum of the tarball_path with the .checksum file in the dest_dir and return False if it matches. The untar of the package happens only if the checksums do not match.
-
update_checksum
(pkg_path)[source]¶ Update the checksum of the package in the packages’ checksum file. This method is called whenever a package is fetched just to be sure that the checksums in the local file are the latest. pkg_path : The complete path to the package file.
-
upload_pkg_dir
(dir_path, upload_path)[source]¶ Upload a full directory. Depending on the upload path, the appropriate method for that protocol is called. Currently this copies the whole tmp package directory to the target directory. This assumes that the web server is running on the same machine where the method is being called from. The upload_path’s files are basically served by that web server.
-
upload_pkg_file
(file_path, upload_path)[source]¶ Upload a single file. Depending on the upload path, the appropriate method for that protocol is called. Currently this simply copies the file to the target directory (but can be extended for other protocols) This assumes that the web server is running on the same machine where the method is being called from. The upload_path’s files are basically served by that web server.
-
upload_pkg_parallel
(pkg_path, upload_path, update_checksum=False)[source]¶ Uploads to a specified upload_path or to all the repos. Also uploads the checksum file to all the repos. pkg_path : The complete path to the package file upload_path : the absolute path where the files are copied to.
if set to ‘None’ assumes ‘all’ repos- update_checksum : If set to False, the checksum file is not
- going to be updated which happens by default. This is necessary for custom packages (like custom kernels and custom tests) that get uploaded which do not need to be part of the checksum file and bloat it.
-
-
class
autotest.client.shared.base_packages.
GitFetcher
(package_manager, repository_url)[source]¶ Bases:
autotest.client.shared.base_packages.RepositoryFetcher
A git based repository fetcher
-
fetch_pkg_file
(filename, dest_path)[source]¶ Fetch a package file and save it to the given destination path
git is an SCM, you can download the test directly. No need to fetch a bz2’d tarball file. However ‘filename’ is <type>-<name>.tar.bz2 break this up and only fetch <name>.
Parameters: - filename (string) – The filename of the package file to fetch.
- dest_path (string) – Destination path to download the file to.
-
git_archive_cmd_pattern
= 'git archive --remote=%s -o %s %s'¶
-
install_pkg_post
(filename, fetch_dir, install_dir, preserve_install_dir=False)[source]¶ Fetcher specific post install
Parameters: - filename (string) – The filename of the package to install
- fetch_dir (string) – The fetched path of the package
- install_dir (string) – The path to install the package to
@preserve_install_dir: Preserve the install directory
-
-
class
autotest.client.shared.base_packages.
HttpFetcher
(package_manager, repository_url)[source]¶ Bases:
autotest.client.shared.base_packages.RepositoryFetcher
Repository Fetcher using HTTP
-
fetch_pkg_file
(filename, dest_path)[source]¶ Fetch a package file from a package repository.
Parameters: - filename (string) – The filename of the package file to fetch.
- dest_path (string) – Destination path to download the file to.
Raises: PackageFetchError – if the fetch failed
-
wget_cmd_pattern
= 'wget --connect-timeout=15 -nv %s -O %s'¶
-
-
class
autotest.client.shared.base_packages.
LocalFilesystemFetcher
(package_manager, repository_url)[source]¶ Bases:
autotest.client.shared.base_packages.RepositoryFetcher
-
fetch_pkg_file
(filename, dest_path)[source]¶ Fetch a package file from a package repository.
Parameters: - filename (string) – The filename of the package file to fetch.
- dest_path (string) – Destination path to download the file to.
Raises: PackageFetchError – if the fetch failed
-
-
class
autotest.client.shared.base_packages.
RepositoryFetcher
(package_manager, repository_url)[source]¶ Bases:
object
Base class with common functionality for repository fetchers
-
fetch_pkg_file
(filename, dest_path)[source]¶ Fetch a package file from a package repository.
Parameters: - filename (string) – The filename of the package file to fetch.
- dest_path (string) – Destination path to download the file to.
Raises: PackageFetchError – if the fetch failed
-
install_pkg_post
(filename, fetch_dir, install_dir, preserve_install_dir=False)[source]¶ Fetcher specific post install
Parameters: - filename (string) – The filename of the package to install
- fetch_dir (string) – The fetched path of the package
- install_dir (string) – The path to install the package to
@preserve_install_dir: Preserve the install directory
-
install_pkg_setup
(name, fetch_dir, install)[source]¶ Install setup for a package based on fetcher type.
Parameters: - name (string) – The filename to be munged
- fetch_dir (string) – The destination path to be munged
- install (boolean) – Whether this is be called from the install path or not
Returns: tuple with (name, fetch_dir)
-
url
= None¶
-
-
autotest.client.shared.base_packages.
check_diskspace
(repo, min_free=None)[source]¶ Check if the remote directory over at the pkg repo has available diskspace
If the amount of free space is not supplied, it is taken from the global configuration file, section [PACKAGES], key ‘mininum_free_space’. The unit used are in SI, that is, 1 GB = 10**9 bytes.
Parameters: repo (string) – a remote package repo URL
Param: min_free mininum amount of free space, in GB (10**9 bytes)
Raises: - error.RepoUnknownError – general repository error condition
- error.RepoDiskFullError – repository does not have at least the requested amount of free disk space.
-
autotest.client.shared.base_packages.
check_write
(repo)[source]¶ Checks that the remote repository directory is writable
Parameters: repo (string) – a remote package repo URL Raises: error.RepoWriteError – repository write error
-
autotest.client.shared.base_packages.
create_directory
(repo)[source]¶ Create a directory over at the remote repository
Parameters: repo (string) – the repo URL containing the remote directory path Returns: a CmdResult object or None
-
autotest.client.shared.base_packages.
has_pbzip2
()[source]¶ Check if parallel bzip2 is available on this system.
Returns: True if pbzip2 is available, False otherwise
-
autotest.client.shared.base_packages.
parse_ssh_path
(repo)[source]¶ Parse an SSH url
Parameters: repo (string) – a repo uri like ssh://xx@xx/path/to/ Returns: tuple with (host, remote_path)
-
autotest.client.shared.base_packages.
repo_run_command
(repo, cmd, ignore_status=False, cd=True)[source]¶ Run a command relative to the repo path
This is basically a utils.run() wrapper that sets itself in a repo directory if it is appropriate, so parameters such as cmd and ignore_status are passed along to it.
Parameters: - repo (string) – a repository url
- cmd (string) – the command to be executed. This is passed along to utils.run()
- ignore_status (boolean) – do not raise an exception, no matter what the exit code of the command is.
- cd (boolean) – wether to change the working directory to the repo directory before running the specified command.
Returns: a CmdResult object or None
Raises: CmdError – the exit code of the command execution was not 0
-
autotest.client.shared.base_packages.
trim_custom_directories
(repo, older_than_days=None)[source]¶ Remove old files from the remote repo directory
The age of the files, if not provided by the older_than_days parameter is taken from the global configuration file, at section [PACKAGES], configuration item ‘custom_max_age’.
Parameters: repo (string) – a remote package repo URL
base_syncdata
Module¶
-
class
autotest.client.shared.base_syncdata.
SyncData
(masterid, hostid, hosts, session_id=None, listen_server=None, port=13234, tmpdir=None)[source]¶ Bases:
object
Provides data synchronization between hosts.
Transferred data is pickled and sent to all destination points. If there is no listen server it will create a new one. If multiple hosts wants to communicate with each other, then communications are identified by session_id.
-
class
autotest.client.shared.base_syncdata.
SyncListenServer
(address='', port=13234, tmpdir=None)[source]¶ Bases:
object
-
class
autotest.client.shared.base_syncdata.
TempDir
(tmpdir=None)[source]¶ Bases:
autotest.client.shared.autotemp.tempdir
TempDir class is tempdir for predefined tmpdir.
boottool
Module¶
boottool client-side module.
This module provides an API for client side tests that need to manipulate boot entries. It’s based on the rewrite of boottool, now python and grubby based. It aims to be keep API compatibility with the older version, except from XEN support which has been removed. We’ll gladly accept patches that provide full coverage for this mode/feature.
Copyright 2009 Google Inc. Copyright 2012 Red Hat, Inc.
Released under the GPL v2
-
class
autotest.client.shared.boottool.
boottool
(path=None)[source]¶ Bases:
autotest.client.tools.boottool.Grubby
Client site side boottool wrapper.
Inherits all functionality from boottool(.py) CLI app (lazily).
check_version
Module¶
-
class
autotest.client.shared.check_version.
check_python_version
[source]¶ Bases:
autotest.client.shared.check_version.site_check_python_version
,autotest.client.shared.base_check_version.base_check_python_version
common
Module¶
control_data
Module¶
-
class
autotest.client.shared.control_data.
ControlData
(vars, path, raise_warnings=False)[source]¶ Bases:
object
distro
Module¶
This module provides the client facilities to detect the Linux Distribution it’s running under.
This is a replacement for the get_os_vendor() function from the utils module.
-
class
autotest.client.shared.distro.
LinuxDistro
(name, version, release, arch)[source]¶ Bases:
object
Simple collection of information for a Linux Distribution
-
class
autotest.client.shared.distro.
Probe
[source]¶ Bases:
object
Probes the machine and does it best to confirm it’s the right distro
-
CHECK_FILE
= None¶ Points to a file that can determine if this machine is running a given Linux Distribution. This servers a first check that enables the extra checks to carry on.
-
CHECK_FILE_CONTAINS
= None¶ Sets the content that should be checked on the file pointed to by
CHECK_FILE_EXISTS
. Leave it set to None (its default) to check only if the file exists, and not check its contents
-
CHECK_FILE_DISTRO_NAME
= None¶ The name of the Linux Distribution to be returned if the file defined by
CHECK_FILE_EXISTS
exist.
-
CHECK_VERSION_REGEX
= None¶ A regular expresion that will be run on the file pointed to by
CHECK_FILE_EXISTS
-
check_name_for_file
()[source]¶ Checks if this class will look for a file and return a distro
The conditions that must be true include the file that identifies the distro file being set (
CHECK_FILE
) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME
)
-
check_name_for_file_contains
()[source]¶ Checks if this class will look for text on a file and return a distro
The conditions that must be true include the file that identifies the distro file being set (
CHECK_FILE
), the text to look for inside the distro file (CHECK_FILE_CONTAINS
) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME
)
-
get_distro
()[source]¶ Returns the
LinuxDistro
this probe detected
-
name_for_file
()[source]¶ Get the distro name if the
CHECK_FILE
is set and exists
-
name_for_file_contains
()[source]¶ Get the distro if the
CHECK_FILE
is set and has content
-
-
autotest.client.shared.distro.
register_probe
(probe_class)[source]¶ Register a probe to be run during autodetection
-
autotest.client.shared.distro.
detect
()[source]¶ Attempts to detect the Linux Distribution running on this machine
Returns: the detected LinuxDistro
orUNKNOWN_DISTRO
Return type: LinuxDistro
distro_def
Module¶
This module defines a structure and portable format for relevant information on Linux Distributions in such a way that information about known distros can be packed and distributed.
Please note that this module deals with Linux Distributions not necessarily installed on the running system.
-
autotest.client.shared.distro_def.
save
(linux_distro, path)[source]¶ Saves the linux_distro to an external file format
Parameters: Returns: None
-
autotest.client.shared.distro_def.
load
(path)[source]¶ Loads the distro from an external file
Parameters: path (str) – the location for the input file Returns: an DistroDef
instanceReturn type: DistroDef
-
autotest.client.shared.distro_def.
load_from_tree
(name, version, release, arch, package_type, path)[source]¶ Loads a DistroDef from an installable tree
Parameters: - name (str) – a short name that precisely distinguishes this Linux Distribution among all others.
- version (str) – the major version of the distribution. Usually this is a single number that denotes a large development cycle and support file.
- release (str) – the release or minor version of the distribution. Usually this is also a single number, that is often omitted or starts with a 0 when the major version is initially release. It’s ofter associated with a shorter development cycle that contains incremental a collection of improvements and fixes.
- arch (str) – the main target for this Linux Distribution. It’s common for some architectures to ship with packages for previous and still compatible architectures, such as it’s the case with Intel/AMD 64 bit architecture that support 32 bit code. In cases like this, this should be set to the 64 bit architecture name.
- package_type (str) – one of the available package info loader types
- path (str) – top level directory of the distro installation tree files
-
class
autotest.client.shared.distro_def.
SoftwarePackage
(name, version, release, checksum, arch)[source]¶ Bases:
object
Definition of relevant information on a software package
-
class
autotest.client.shared.distro_def.
DistroDef
(name, version, release, arch)[source]¶ Bases:
autotest.client.shared.distro.LinuxDistro
More complete information on a given Linux Distribution
-
software_packages
= None¶ All the software packages that ship with this Linux distro
-
software_packages_type
= None¶ A simple text that denotes the software type that makes this distro
-
-
autotest.client.shared.distro_def.
DISTRO_PKG_INFO_LOADERS
= {'deb': <class 'autotest.client.shared.distro_def.DistroPkgInfoLoaderDeb'>, 'rpm': <class 'autotest.client.shared.distro_def.DistroPkgInfoLoaderRpm'>}¶ the type of distro that will determine what loader will be used
enum
Module¶
Generic enumeration support.
-
class
autotest.client.shared.enum.
Enum
(*names, **kwargs)[source]¶ Bases:
object
Utility class to implement Enum-like functionality.
>>> e = Enum('String one', 'String two') >>> e.STRING_ONE 0 >>> e.STRING_TWO 1 >>> e.choices() [(0, 'String one'), (1, 'String two')] >>> e.get_value('String one') 0 >>> e.get_string(0) 'String one'
>>> e = Enum('Hello', 'Goodbye', string_values=True) >>> e.HELLO, e.GOODBYE ('Hello', 'Goodbye')
>>> e = Enum('One', 'Two', start_value=1) >>> e.ONE 1 >>> e.TWO 2
error
Module¶
Internal global error types
-
autotest.client.shared.error.
context_aware
(fn)[source]¶ A decorator that must be applied to functions that call context().
-
autotest.client.shared.error.
context
(s='', log=None)[source]¶ Set the context for the currently executing function and optionally log it.
Parameters: - s – A string. If not provided, the context for the current function will be cleared.
- log – A logging function to pass the context message to. If None, no function will be called.
-
autotest.client.shared.error.
get_context
()[source]¶ Return the current context (or None if none is defined).
-
autotest.client.shared.error.
exception_context
(e)[source]¶ Return the context of a given exception (or None if none is defined).
-
exception
autotest.client.shared.error.
AutoservHostIsShuttingDownError
[source]¶ Bases:
autotest.client.shared.error.AutoservHostError
Host is shutting down
-
exception
autotest.client.shared.error.
TestBug
[source]¶ Bases:
autotest.client.shared.error.TestBaseException
Indicates that the test failed, but the fail was expected.
-
exit_status
= 'BUG'¶
-
-
exception
autotest.client.shared.error.
AutoservHardwareRepairRequiredError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Exception class raised during repairs to indicate that a hardware repair is going to be necessary.
-
exception
autotest.client.shared.error.
RepoWriteError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when packager cannot write to a repo’s desitnation
-
exception
autotest.client.shared.error.
AutoservUnsupportedError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Error raised when you try to use an unsupported optional feature
-
exception
autotest.client.shared.error.
CmdError
(command, result_obj, additional_text=None)[source]¶ Bases:
autotest.client.shared.error.TestError
Indicates that a command failed, is fatal to the test unless caught.
-
exception
autotest.client.shared.error.
AutotestError
[source]¶ Bases:
exceptions.Exception
The parent of all errors deliberately thrown within the client code.
-
exception
autotest.client.shared.error.
RepoDiskFullError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when the destination for packages is full
-
exception
autotest.client.shared.error.
AutoservRebootError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Error occurred while rebooting a machine
-
exception
autotest.client.shared.error.
TestWarn
[source]¶ Bases:
autotest.client.shared.error.TestBaseException
Indicates that bad things (may) have happened, but not an explicit failure.
-
exit_status
= 'WARN'¶
-
-
exception
autotest.client.shared.error.
PackageInstallError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when there is an error installing the package
-
exception
autotest.client.shared.error.
HostInstallProfileError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates the machine failed to have a profile assigned.
-
exception
autotest.client.shared.error.
PackageError
[source]¶ Bases:
autotest.client.shared.error.TestError
Indicates an error trying to perform a package operation.
-
exception
autotest.client.shared.error.
AutotestHostRunError
(description, result_obj)[source]¶ Bases:
autotest.client.shared.error.HostRunErrorMixIn
,autotest.client.shared.error.AutotestError
-
exception
autotest.client.shared.error.
UnhandledTestFail
(unhandled_exception)[source]¶ Bases:
autotest.client.shared.error.TestFail
Indicates an unhandled fail in a test.
-
exception
autotest.client.shared.error.
BarrierAbortError
[source]¶ Bases:
autotest.client.shared.error.BarrierError
Indicate that the barrier was explicitly aborted by a member.
-
exception
autotest.client.shared.error.
AutoservSubcommandError
(func, exit_code)[source]¶ Bases:
autotest.client.shared.error.AutoservError
Indicates an error while executing a (forked) subcommand
-
exception
autotest.client.shared.error.
NetCommunicationError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicate that network communication was broken.
-
exception
autotest.client.shared.error.
AutoservShutdownError
[source]¶ Bases:
autotest.client.shared.error.AutoservRebootError
Error occurred during shutdown of machine
-
exception
autotest.client.shared.error.
PackageRemoveError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when there is an error removing the package
-
exception
autotest.client.shared.error.
UnhandledTestError
(unhandled_exception)[source]¶ Bases:
autotest.client.shared.error.TestError
Indicates an unhandled error in a test.
-
exception
autotest.client.shared.error.
DataSyncError
[source]¶ Bases:
autotest.client.shared.error.NetCommunicationError
Indicates problem during synchronization data over network.
-
exception
autotest.client.shared.error.
AutoservHostError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Error reaching a host
-
exception
autotest.client.shared.error.
TestBaseException
[source]¶ Bases:
autotest.client.shared.error.AutotestError
The parent of all test exceptions.
-
exit_status
= 'NEVER_RAISE_THIS'¶
-
-
exception
autotest.client.shared.error.
TestNAError
[source]¶ Bases:
autotest.client.shared.error.TestBaseException
Indictates that the test is Not Applicable. Should be thrown when various conditions are such that the test is inappropriate.
-
exit_status
= 'TEST_NA'¶
-
-
exception
autotest.client.shared.error.
AutoservHardwareHostError
[source]¶ Bases:
autotest.client.shared.error.AutoservHostError
Found hardware problems with the host
-
exception
autotest.client.shared.error.
AutoservSSHTimeout
[source]¶ Bases:
autotest.client.shared.error.AutoservError
SSH experienced a connection timeout
-
exception
autotest.client.shared.error.
InstallError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates an installation error which Terminates and fails the job.
-
exception
autotest.client.shared.error.
AutoservDiskFullHostError
(path, want_gb, free_space_gb)[source]¶ Bases:
autotest.client.shared.error.AutoservHostError
Not enough free disk space on host
-
exception
autotest.client.shared.error.
AutoservInstallError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Error occurred while installing autotest on a host
-
exception
autotest.client.shared.error.
TestError
[source]¶ Bases:
autotest.client.shared.error.TestBaseException
Indicates that something went wrong with the test harness itself.
-
exit_status
= 'ERROR'¶
-
-
exception
autotest.client.shared.error.
AutoservVirtError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Vitualization related error
-
exception
autotest.client.shared.error.
BarrierError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates an error happened during a barrier operation.
-
exception
autotest.client.shared.error.
AutotestRunError
[source]¶ Bases:
autotest.client.shared.error.AutotestError
Indicates a problem running server side control files.
-
exception
autotest.client.shared.error.
RepoError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when a repo isn’t working in some way
-
exception
autotest.client.shared.error.
PackagingError
[source]¶ Bases:
autotest.client.shared.error.AutotestError
Abstract error class for all packaging related errors.
-
exception
autotest.client.shared.error.
RepoUnknownError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when packager cannot write to a repo’s desitnation
-
exception
autotest.client.shared.error.
UnhandledJobError
(unhandled_exception)[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates an unhandled error in a job.
-
exception
autotest.client.shared.error.
TestFail
[source]¶ Bases:
autotest.client.shared.error.TestBaseException
Indicates that the test failed, but the job will not continue.
-
exit_status
= 'FAIL'¶
-
-
exception
autotest.client.shared.error.
JobError
[source]¶ Bases:
autotest.client.shared.error.AutotestError
Indicates an error which terminates and fails the whole job (ABORT).
-
exception
autotest.client.shared.error.
AutoservRunError
(description, result_obj)[source]¶ Bases:
autotest.client.shared.error.HostRunErrorMixIn
,autotest.client.shared.error.AutoservError
-
exception
autotest.client.shared.error.
PackageFetchError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when there is an error fetching the package
-
exception
autotest.client.shared.error.
PackageUploadError
[source]¶ Bases:
autotest.client.shared.error.PackagingError
Raised when there is an error uploading the package
-
exception
autotest.client.shared.error.
AutoservHardwareRepairRequestedError
[source]¶ Bases:
autotest.client.shared.error.AutoservError
Exception class raised from Host.repair_full() (or overrides) when software repair fails but it successfully managed to request a hardware repair (by notifying the staff, sending mail, etc)
-
exception
autotest.client.shared.error.
HostRunErrorMixIn
(description, result_obj)[source]¶ Bases:
exceptions.Exception
Indicates a problem in the host run() function raised from client code. Should always be constructed with a tuple of two args (error description (str), run result object). This is a common class mixed in to create the client and server side versions of it.
-
exception
autotest.client.shared.error.
HarnessError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates problem with the harness.
-
exception
autotest.client.shared.error.
AutoservNotMountedHostError
[source]¶ Bases:
autotest.client.shared.error.AutoservHostError
Found unmounted partitions that should be mounted
-
exception
autotest.client.shared.error.
AutoservSshPermissionDeniedError
(description, result_obj)[source]¶ Bases:
autotest.client.shared.error.AutoservRunError
Indicates that a SSH permission denied error was encountered.
-
exception
autotest.client.shared.error.
HostInstallTimeoutError
[source]¶ Bases:
autotest.client.shared.error.JobError
Indicates the machine failed to be installed after the predetermined timeout.
-
exception
autotest.client.shared.error.
AutoservSshPingHostError
[source]¶ Bases:
autotest.client.shared.error.AutoservHostError
SSH ping failed
-
exception
autotest.client.shared.error.
AutotestTimeoutError
[source]¶ Bases:
autotest.client.shared.error.AutotestError
This exception is raised when an autotest test exceeds the timeout parameter passed to run_timed_test and is killed.
git
Module¶
Code that helps to deal with content from git repositories
-
class
autotest.client.shared.git.
GitRepoHelper
(uri, branch='master', lbranch=None, commit=None, destination_dir=None, base_uri=None)[source]¶ Bases:
object
Helps to deal with git repos, mostly fetching content from a repo
-
checkout
(branch=None, commit=None)[source]¶ Performs a git checkout for a given branch and start point (commit)
Parameters: - branch – Remote branch name.
- commit – Specific commit hash.
-
execute
()[source]¶ Performs all steps necessary to initialize and download a git repo.
This includes the init, fetch and checkout steps in one single utility method.
-
-
autotest.client.shared.git.
get_repo
(uri, branch='master', lbranch=None, commit=None, destination_dir=None, base_uri=None)[source]¶ Utility function that retrieves a given git code repository.
Parameters: - uri (string) – git repository url
- branch (string) – git remote branch
- destination_dir (string) – path of a dir where to save downloaded code
- commit (string) – specific commit to download
- lbranch (string) – git local branch name, if different from remote
- uri – a closer, usually local, git repository url from where to fetch content first from
host_protections
Module¶
host_queue_entry_states
Module¶
This module contains the status enums for use by HostQueueEntrys in the database. It is a stand alone module as these status strings are needed from various disconnected pieces of code that should not depend on everything that autotest.frontend.afe.models depends on such as RPC clients.
iscsi
Module¶
Basic iscsi support for Linux host with the help of commands iscsiadm and tgtadm.
This include the basic operates such as login and get device name by target name. And it can support the real iscsi access and emulated iscsi in localhost then access it.
-
class
autotest.client.shared.iscsi.
Iscsi
(params, root_dir='/tmp')[source]¶ Bases:
object
Basic iscsi support class. Will handle the emulated iscsi export and access to both real iscsi and emulated iscsi device.
-
autotest.client.shared.iscsi.
iscsi_discover
(portal_ip)[source]¶ Query from iscsi server for available targets
Parameters: portal_ip – Ip for iscsi server
iso9660
Module¶
Basic ISO9660 file-system support.
This code does not attempt (so far) to implement code that knows about ISO9660 internal structure. Instead, it uses commonly available support either in userspace tools or on the Linux kernel itself (via mount).
-
autotest.client.shared.iso9660.
iso9660
(path)[source]¶ Checks the avaiable tools on a system and chooses class accordingly
This is a convinience function, that will pick the first avaialable iso9660 capable tool.
Parameters: path (str) – path to an iso9660 image file Returns: an instance of any iso9660 capable tool Return type: Iso9660IsoInfo
,Iso9660IsoRead
,Iso9660Mount
or None
-
class
autotest.client.shared.iso9660.
Iso9660IsoInfo
(path)[source]¶ Bases:
autotest.client.shared.iso9660.BaseIso9660
Represents a ISO9660 filesystem
This implementation is based on the cdrkit’s isoinfo tool
-
class
autotest.client.shared.iso9660.
Iso9660IsoRead
(path)[source]¶ Bases:
autotest.client.shared.iso9660.BaseIso9660
Represents a ISO9660 filesystem
This implementation is based on the libcdio’s iso-read tool
jsontemplate
Module¶
Python implementation of json-template.
JSON Template is a minimal and powerful templating language for transforming a JSON dictionary to arbitrary text.
To use this module, you will typically use the Template constructor, and catch various exceptions thrown. You may also want to use the FromFile/FromString methods, which allow Template constructor options to be embedded in the template string itself.
Other functions are exposed for tools which may want to process templates.
-
exception
autotest.client.shared.jsontemplate.
Error
[source]¶ Bases:
exceptions.Exception
Base class for all exceptions in this module.
Thus you can “except jsontemplate.Error: to catch all exceptions thrown by this module.
-
exception
autotest.client.shared.jsontemplate.
CompilationError
[source]¶ Bases:
autotest.client.shared.jsontemplate.Error
Base class for errors that happen during the compilation stage.
-
exception
autotest.client.shared.jsontemplate.
EvaluationError
(msg, original_exception=None)[source]¶ Bases:
autotest.client.shared.jsontemplate.Error
Base class for errors that happen when expanding the template.
This class of errors generally involve the data dictionary or the execution of the formatters.
-
exception
autotest.client.shared.jsontemplate.
BadFormatter
[source]¶ Bases:
autotest.client.shared.jsontemplate.CompilationError
A bad formatter was specified, e.g. {variable|BAD}
-
exception
autotest.client.shared.jsontemplate.
BadPredicate
[source]¶ Bases:
autotest.client.shared.jsontemplate.CompilationError
A bad predicate was specified, e.g. {.BAD?}
-
exception
autotest.client.shared.jsontemplate.
MissingFormatter
[source]¶ Bases:
autotest.client.shared.jsontemplate.CompilationError
Raised when formatters are required, and a variable is missing a formatter.
-
exception
autotest.client.shared.jsontemplate.
ConfigurationError
[source]¶ Bases:
autotest.client.shared.jsontemplate.CompilationError
Raised when the Template options are invalid and it can’t even be compiled.
-
exception
autotest.client.shared.jsontemplate.
TemplateSyntaxError
[source]¶ Bases:
autotest.client.shared.jsontemplate.CompilationError
Syntax error in the template text.
-
exception
autotest.client.shared.jsontemplate.
UndefinedVariable
(msg, original_exception=None)[source]¶ Bases:
autotest.client.shared.jsontemplate.EvaluationError
The template contains a variable not defined by the data dictionary.
-
autotest.client.shared.jsontemplate.
CompileTemplate
(template_str, builder=None, meta='{}', format_char='|', more_formatters=<function <lambda>>, more_predicates=<function <lambda>>, default_formatter='str')[source]¶ Compile the template string, calling methods on the ‘program builder’.
- Args:
- template_str: The template string. It should not have any compilation
- options in the header – those are parsed by FromString/FromFile
- builder: The interface of _ProgramBuilder isn’t fixed. Use at your own
- risk.
meta: The metacharacters to use, e.g. ‘{}’, ‘[]’.
- more_formatters:
- Something that can map format strings to formatter functions. One of:
- A plain dictionary of names -> functions e.g. {‘html’: cgi.escape}
- A higher-order function which takes format strings and returns formatter functions. Useful for when formatters have parsed arguments.
- A FunctionRegistry instance for the most control. This allows formatters which takes contexts as well.
- more_predicates:
- Like more_formatters, but for predicates.
- default_formatter: The formatter to use for substitutions that are missing a
- formatter. The ‘str’ formatter the “default default” – it just tries to convert the context value to a string in some unspecified manner.
- Returns:
- The compiled program (obtained from the builder)
- Raises:
- The various subclasses of CompilationError. For example, if default_formatter=None, and a variable is missing a formatter, then MissingFormatter is raised.
This function is public so it can be used by other tools, e.g. a syntax checking tool run before submitting a template to source control.
-
autotest.client.shared.jsontemplate.
FromString
(s, more_formatters=<function <lambda>>, _constructor=None)[source]¶ Like FromFile, but takes a string.
-
autotest.client.shared.jsontemplate.
FromFile
(f, more_formatters=<function <lambda>>, _constructor=None)[source]¶ Parse a template from a file, using a simple file format.
This is useful when you want to include template options in a data file, rather than in the source code.
The format is similar to HTTP or E-mail headers. The first lines of the file can specify template options, such as the metacharacters to use. One blank line must separate the options from the template body.
Example:
default-formatter: none meta: {{}} format-char: : <blank line required> Template goes here: {{variable:html}}- Args:
- f: A file handle to read from. Caller is responsible for opening and closing it.
-
class
autotest.client.shared.jsontemplate.
Template
(template_str, builder=None, undefined_str=None, **compile_options)[source]¶ Bases:
object
Represents a compiled template.
Like many template systems, the template string is compiled into a program, and then it can be expanded any number of times. For example, in a web app, you can compile the templates once at server startup, and use the expand() method at request handling time. expand() uses the compiled representation.
There are various options for controlling parsing – see CompileTemplate. Don’t go crazy with metacharacters. {}, [], {{}} or <> should cover nearly any circumstance, e.g. generating HTML, CSS XML, JavaScript, C programs, text files, etc.
-
expand
(*args, **kwargs)[source]¶ Expands the template with the given data dictionary, returning a string.
This is a small wrapper around render(), and is the most convenient interface.
- Args:
- The JSON data dictionary. Like the builtin dict() constructor, it can take a single dictionary as a positional argument, or arbitrary keyword arguments.
- Returns:
- The return value could be a str() or unicode() instance, depending on the the type of the template string passed in, and what the types the strings in the dictionary are.
-
kernel_versions
Module¶
log
Module¶
-
autotest.client.shared.log.
log_and_ignore_errors
(msg)[source]¶ A decorator for wrapping functions in a ‘log exception and ignore’ try-except block.
-
autotest.client.shared.log.
record
(fn)[source]¶ Generic method decorator for logging calls under the assumption that return=GOOD, exception=FAIL. The method determines parameters as:
subdir = self.subdir if it exists, or None operation = “class name”.”method name” status = None on GOOD, str(exception) on FAILThe object using this method must have a job attribute for the logging to actually occur, otherwise the logging will silently fail.
Logging can explicitly be disabled for a call by passing a logged=False parameter
logging_config
Module¶
-
class
autotest.client.shared.logging_config.
AllowBelowSeverity
(level)[source]¶ Bases:
logging.Filter
Allows only records less severe than a given level (the opposite of what the normal logging level filtering does.
logging_manager
Module¶
-
class
autotest.client.shared.logging_manager.
FdRedirectionLoggingManager
[source]¶ Bases:
autotest.client.shared.logging_manager.LoggingManager
A simple extension of LoggingManager to use FdRedirectionStreamManagers, so that managed streams have their underlying FDs redirected.
-
STREAM_MANAGER_CLASS
¶ alias of
_FdRedirectionStreamManager
-
-
class
autotest.client.shared.logging_manager.
LoggingFile
(prefix='', level=10, logger=<logging.RootLogger object>)[source]¶ Bases:
object
File-like object that will receive messages pass them to the logging infrastructure in an appropriate way.
-
class
autotest.client.shared.logging_manager.
LoggingManager
[source]¶ Bases:
object
Manages a stack of logging configurations, allowing clients to conveniently add and remove logging destinations. Also keeps a list of StreamManagers to easily direct streams into the logging module.
-
STREAM_MANAGER_CLASS
¶ alias of
_StreamManager
-
logging_config_object
= None¶
-
manage_stream
(stream, level, stream_setter)[source]¶ Tells this manager to manage the given stream. All data written to the stream will be directed to the logging module instead. Must be called before start_logging().
Parameters: - stream – stream to manage
- level – level to log data written to this stream
- stream_setter – function to set the stream to a new object
-
-
class
autotest.client.shared.logging_manager.
SortingLoggingFile
(prefix='', level_list=[('ERROR', 40), ('WARN', 30), ('INFO', 20), ('DEBUG', 10)], logger=<logging.RootLogger object>)[source]¶ Bases:
autotest.client.shared.logging_manager.LoggingFile
File-like object that will receive messages and pass them to the logging infrastructure. It decides where to pass each line by applying a regex to it and seeing which level it matched.
-
autotest.client.shared.logging_manager.
configure_logging
(logging_config, **kwargs)[source]¶ Configure the logging module using the specific configuration object, which should be an instance of logging_config.LoggingConfig (usually of a subclass). Any keyword args will be passed to the object’s configure_logging() method.
Every entry point should call this method at application startup.
-
autotest.client.shared.logging_manager.
do_not_report_as_logging_caller
(func)[source]¶ Decorator to annotate functions we will tell logging not to log.
-
autotest.client.shared.logging_manager.
get_logging_manager
(manage_stdout_and_stderr=False, redirect_fds=False)[source]¶ Create a LoggingManager that’s managing sys.stdout and sys.stderr.
Every entry point that wants to capture stdout/stderr and/or use LoggingManager to manage a stack of destinations should call this method at application startup.
magic
Module¶
Library used to determine a file MIME type by its magic number, it doesn’t have any external dependencies. Based on work of Jason Petrone (jp_py@jsnp.net), adapted to autotest.
- Command Line Usage: Running as ‘python magic.py file_path’ will print a
- mime string (or just a description) of the file present on file_path.
- API Usage:
- magic.guess_type(file_path) - Returns a description of what the file on path ‘file’ contains. This function name was chosen due to a similar function on python standard library ‘mimetypes’.
@license: GPL v2 :copyright: Jason Petrone (jp_py@jsnp.net) 2000 :copyright: Lucas Meneghel Rodrigues (lmr@redhat.com) 2010 @see: http://www.jsnp.net/code/magic.py
-
class
autotest.client.shared.magic.
MagicTest
(offset, t, op, value, msg, mask=None)[source]¶ Bases:
object
Compile a magic database entry so it can be compared with data read from files.
mail
Module¶
Notification email library.
Aims to replace a bunch of different email module wrappers previously used.
-
class
autotest.client.shared.mail.
EmailNotificationManager
(module='scheduler')[source]¶ Bases:
object
Email notification facility, for use in things like the autotest scheduler.
This facility can use values defined in the autotest settings (global_config.ini) to conveniently send notification emails to the admin of an autotest module.
-
enqueue_exception_admin
(reason)[source]¶ Enqueue an email containing an exception to the test grid admin.
-
-
autotest.client.shared.mail.
send
(from_address, to_addresses, cc_addresses, subject, body, smtp_info, html=None)[source]¶ Send out an email.
- Args:
from_address: The email address to put in the “From:” field. to_addresses: Either a single string or an iterable of
strings to put in the “To:” field of the email.- cc_addresses: Either a single string of an iterable of
- strings to put in the “Cc:” field of the email.
subject: The email subject. body: The body of the email. there’s no special
handling of encoding here, so it’s safest to stick to 7-bit ASCII text.smtp_info: Dictionary with SMTP info. html: Optional HTML content of the message.
mock
Module¶
-
class
autotest.client.shared.mock.
Mock
(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]¶ Bases:
autotest.client.shared.mock.CallableMixin
,autotest.client.shared.mock.NonCallableMock
Create a new Mock object. Mock takes several optional arguments that specify the behaviour of the Mock object:
spec: This can be either a list of strings or an existing object (a class or instance) that acts as the specification for the mock object. If you pass in an object then a list of strings is formed by calling dir on the object (excluding unsupported magic attributes and methods). Accessing any attribute not in this list will raise an AttributeError.
If spec is an object (rather than a list of strings) then mock.__class__ returns the class of the spec object. This allows mocks to pass isinstance tests.
spec_set: A stricter variant of spec. If used, attempting to set or get an attribute on the mock that isn’t on the object passed as spec_set will raise an AttributeError.
side_effect: A function to be called whenever the Mock is called. See the side_effect attribute. Useful for raising exceptions or dynamically changing return values. The function is called with the same arguments as the mock, and unless it returns DEFAULT, the return value of this function is used as the return value.
Alternatively side_effect can be an exception class or instance. In this case the exception will be raised when the mock is called.
If side_effect is an iterable then each call to the mock will return the next value from the iterable. If any of the members of the iterable are exceptions they will be raised instead of returned.
return_value: The value returned when the mock is called. By default this is a new Mock (created on first access). See the return_value attribute.
wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a Mock object that wraps the corresponding attribute of the wrapped object (so attempting to access an attribute that doesn’t exist will raise an AttributeError).
If the mock has an explicit return_value set then calls are not passed to the wrapped object and the return_value is returned instead.
name: If the mock has a name then it will be used in the repr of the mock. This can be useful for debugging. The name is propagated to child mocks.
Mocks can also be called with arbitrary keyword arguments. These will be used to set attributes on the mock after it is created.
-
class
autotest.client.shared.mock.
MagicMock
(*args, **kw)[source]¶ Bases:
autotest.client.shared.mock.MagicMixin
,autotest.client.shared.mock.Mock
MagicMock is a subclass of Mock with default implementations of most of the magic methods. You can use MagicMock without having to configure the magic methods yourself.
If you use the spec or spec_set arguments then only magic methods that exist in the spec will be created.
Attributes and the return value of a MagicMock will also be MagicMocks.
-
autotest.client.shared.mock.
patch
(target, new=sentinel.DEFAULT, spec=None, create=False, spec_set=None, autospec=None, new_callable=None, **kwargs)[source]¶ patch acts as a function decorator, class decorator or a context manager. Inside the body of the function or with statement, the target is patched with a new object. When the function/with statement exits the patch is undone.
If new is omitted, then the target is replaced with a MagicMock. If patch is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function. If patch is used as a context manager the created mock is returned by the context manager.
target should be a string in the form ‘package.module.ClassName’. The target is imported and the specified object replaced with the new object, so the target must be importable from the environment you are calling patch from. The target is imported when the decorated function is executed, not at decoration time.
The spec and spec_set keyword arguments are passed to the MagicMock if patch is creating one for you.
In addition you can pass spec=True or spec_set=True, which causes patch to pass in the object being mocked as the spec/spec_set object.
new_callable allows you to specify a different class, or callable object, that will be called to create the new object. By default MagicMock is used.
A more powerful form of spec is autospec. If you set autospec=True then the mock with be created with a spec from the object being replaced. All attributes of the mock will also have the spec of the corresponding attribute of the object being replaced. Methods and functions being mocked will have their arguments checked and will raise a TypeError if they are called with the wrong signature. For mocks replacing a class, their return value (the ‘instance’) will have the same spec as the class.
Instead of autospec=True you can pass autospec=some_object to use an arbitrary object as the spec instead of the one being replaced.
By default patch will fail to replace attributes that don’t exist. If you pass in create=True, and the attribute doesn’t exist, patch will create the attribute for you when the patched function is called, and delete it again afterwards. This is useful for writing tests against attributes that your production code creates at runtime. It is off by by default because it can be dangerous. With it switched on you can write passing tests against APIs that don’t actually exist!
Patch can be used as a TestCase class decorator. It works by decorating each test method in the class. This reduces the boilerplate code when your test methods share a common patchings set. patch finds tests by looking for method names that start with patch.TEST_PREFIX. By default this is test, which matches the way unittest finds tests. You can specify an alternative prefix by setting patch.TEST_PREFIX.
Patch can be used as a context manager, with the with statement. Here the patching applies to the indented block after the with statement. If you use “as” then the patched object will be bound to the name after the “as”; very useful if patch is creating a mock object for you.
patch takes arbitrary keyword arguments. These will be passed to the Mock (or new_callable) on construction.
patch.dict(…), patch.multiple(…) and patch.object(…) are available for alternate use-cases.
-
autotest.client.shared.mock.
call
¶ A tuple for holding the results of a call to a mock, either in the form (args, kwargs) or (name, args, kwargs).
If args or kwargs are empty then a call tuple will compare equal to a tuple without those values. This makes comparisons less verbose:
_Call(('name', (), {})) == ('name',) _Call(('name', (1,), {})) == ('name', (1,)) _Call(((), {'a': 'b'})) == ({'a': 'b'},)
The _Call object provides a useful shortcut for comparing with call:
_Call(((1, 2), {'a': 3})) == call(1, 2, a=3) _Call(('foo', (1, 2), {'a': 3})) == call.foo(1, 2, a=3)
If the _Call has no name then it will match any name.
-
autotest.client.shared.mock.
create_autospec
(spec, spec_set=False, instance=False, _parent=None, _name=None, **kwargs)[source]¶ Create a mock object using another object as a spec. Attributes on the mock will use the corresponding attribute on the spec object as their spec.
Functions or methods being mocked will have their arguments checked to check that they are called with the correct signature.
If spec_set is True then attempting to set attributes that don’t exist on the spec object will raise an AttributeError.
If a class is used as a spec then the return value of the mock (the instance of the class) will have the same spec. You can use a class as the spec for an instance object by passing instance=True. The returned mock will only be callable if instances of the mock are callable.
create_autospec also takes arbitrary keyword arguments that are passed to the constructor of the created mock.
-
class
autotest.client.shared.mock.
NonCallableMock
(spec=None, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]¶ Bases:
autotest.client.shared.mock.Base
A non-callable version of Mock
-
assert_any_call
(*args, **kwargs)[source]¶ assert the mock has been called with the specified arguments.
The assert passes if the mock has ever been called, unlike assert_called_with and assert_called_once_with that only pass if the call is the most recent one.
-
assert_called_once_with
(*args, **kwargs)[source]¶ assert that the mock was called exactly once and with the specified arguments.
-
assert_called_with
(*args, **kwargs)[source]¶ assert that the mock was called with the specified arguments.
Raises an AssertionError if the args and keyword args passed in are different to the last call to the mock.
-
assert_has_calls
(calls, any_order=False)[source]¶ assert the mock has been called with the specified calls. The mock_calls list is checked for the calls.
If any_order is False (the default) then the calls must be sequential. There can be extra calls before or after the specified calls.
If any_order is True then the calls can be in any order, but they must all appear in mock_calls.
-
attach_mock
(mock, attribute)[source]¶ Attach a mock as an attribute of this one, replacing its name and parent. Calls to the attached mock will be recorded in the method_calls and mock_calls attributes of this one.
-
call_args
¶
-
call_args_list
¶
-
call_count
¶
-
called
¶
-
configure_mock
(**kwargs)[source]¶ Set attributes on the mock through keyword arguments.
Attributes plus return values and side effects can be set on child mocks using standard dot notation and unpacking a dictionary in the method call:
>>> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} >>> mock.configure_mock(**attrs)
-
mock_add_spec
(spec, spec_set=False)[source]¶ Add a spec to a mock. spec can either be an object or a list of strings. Only attributes on the spec can be fetched as attributes from the mock.
If spec_set is True then only attributes on the spec can be set.
-
mock_calls
¶
-
return_value
¶
-
side_effect
¶
-
-
class
autotest.client.shared.mock.
NonCallableMagicMock
(*args, **kw)[source]¶ Bases:
autotest.client.shared.mock.MagicMixin
,autotest.client.shared.mock.NonCallableMock
A version of MagicMock that isn’t callable.
-
autotest.client.shared.mock.
mock_open
(mock=None, read_data='')[source]¶ A helper function to create a mock to replace the use of open. It works for open called directly or used as a context manager.
The mock argument is the mock object to configure. If None (the default) then a MagicMock will be created for you, with the API limited to methods or attributes available on standard file handles.
read_data is a string for the read method of the file handle to return. This is an empty string by default.
-
class
autotest.client.shared.mock.
PropertyMock
(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)[source]¶ Bases:
autotest.client.shared.mock.Mock
A mock intended to be used as a property, or other descriptor, on a class. PropertyMock provides __get__ and __set__ methods so you can specify a return value when it is fetched.
Fetching a PropertyMock instance from an object calls the mock, with no args. Setting it calls the mock with the value being set.
openvswitch
Module¶
-
class
autotest.client.shared.openvswitch.
OpenVSwitch
(tmpdir, db_path=None, db_socket=None, db_pidfile=None, ovs_pidfile=None, dbschema=None, install_prefix=None)[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchSystem
OpenVSwtich class.
-
class
autotest.client.shared.openvswitch.
OpenVSwitchControl
[source]¶ Bases:
object
Class select the best matches control class for installed version of OpenVSwitch.
OpenVSwtich parameters are described in man ovs-vswitchd.conf.db
-
static
convert_version_to_int
(version)[source]¶ Parameters: version – (int) Converted from version string 1.4.0 => int 140
-
static
-
class
autotest.client.shared.openvswitch.
OpenVSwitchControlCli
[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchControl
,autotest.client.shared.utils.VersionableClass
Class select the best matches control class for installed version of OpenVSwitch.
-
class
autotest.client.shared.openvswitch.
OpenVSwitchControlCli_140
[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchControlCli
,autotest.client.shared.utils.VersionableClass
Don’t use this class directly. This class is automatically selected by OpenVSwitchControl.
-
classmethod
is_right_version
(version)[source]¶ Check condition for select control class.
Parameters: version – version of OpenVSwtich
-
classmethod
-
class
autotest.client.shared.openvswitch.
OpenVSwitchControlDB
[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchControl
,autotest.client.shared.utils.VersionableClass
Class select the best matches control class for installed version of OpenVSwitch.
-
class
autotest.client.shared.openvswitch.
OpenVSwitchControlDB_140
[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchControlDB
,autotest.client.shared.utils.VersionableClass
Don’t use this class directly. This class is automatically selected by OpenVSwitchControl.
-
class
autotest.client.shared.openvswitch.
OpenVSwitchSystem
(db_path=None, db_socket=None, db_pidfile=None, ovs_pidfile=None, dbschema=None, install_prefix=None)[source]¶ Bases:
autotest.client.shared.openvswitch.OpenVSwitchControlCli
,autotest.client.shared.openvswitch.OpenVSwitchControlDB
OpenVSwtich class.
-
class
autotest.client.shared.openvswitch.
ServiceManager
[source]¶ Bases:
autotest.client.shared.openvswitch.ServiceManagerInterface
-
class
autotest.client.shared.openvswitch.
ServiceManagerInterface
[source]¶
-
class
autotest.client.shared.openvswitch.
ServiceManagerSystemD
[source]¶ Bases:
autotest.client.shared.openvswitch.ServiceManagerInterface
,autotest.client.shared.utils.VersionableClass
-
class
autotest.client.shared.openvswitch.
ServiceManagerSysvinit
[source]¶ Bases:
autotest.client.shared.openvswitch.ServiceManagerInterface
,autotest.client.shared.utils.VersionableClass
packages
Module¶
-
class
autotest.client.shared.packages.
PackageManager
(pkgmgr_dir, hostname=None, repo_urls=None, upload_paths=None, do_locking=True, run_function=<function run>, run_function_args=[], run_function_dargs={})[source]¶ Bases:
autotest.client.shared.base_packages.BasePackageManager
pidfile
Module¶
profiler_manager
Module¶
-
exception
autotest.client.shared.profiler_manager.
ProfilerNotPresentError
(name, *args, **dargs)[source]¶
-
class
autotest.client.shared.profiler_manager.
profiler_manager
(job)[source]¶ Bases:
object
-
before_start
(test)[source]¶ Override to do any setup needed before actually starting the profilers (this function is called before calling test.before_run_once() and profilers.start() in a profiled run).
-
load_profiler
(profiler, args, dargs)[source]¶ Given a name and args, loads a profiler, initializes it with the required arguments, and returns an instance of it. Raises a ProfilerNotPresentError if the module isn’t found.
-
only
()[source]¶ Returns True if job is supposed to be run only with profiling turned on, False otherwise
-
progressbar
Module¶
Basic text progress bar without fancy curses features
report
Module¶
Module used to parse the autotest job status file and generate a JSON file.
Optionally, we can also generate reports (HTML)
-
exception
autotest.client.shared.report.
InvalidAutotestResultDirError
(directory)[source]¶ Bases:
exceptions.Exception
-
exception
autotest.client.shared.report.
InvalidOutputDirError
(directory)[source]¶ Bases:
exceptions.Exception
-
class
autotest.client.shared.report.
ReportLoggingConfig
(use_console=True)[source]¶ Bases:
autotest.client.shared.logging_config.LoggingConfig
Used with the sole purpose of providing convenient logging setup for this program.
-
class
autotest.client.shared.report.
ReportOptionParser
[source]¶ Bases:
optparse.OptionParser
-
autotest.client.shared.report.
generate_html_report
(results_dir, relative_links=True)[source]¶ Render a job report HTML.
All CSS and javascript are inlined, for more convenience.
Parameters: results_dir – Path to the results directory.
-
autotest.client.shared.report.
generate_json_file
(results_dir, relative_links=True)[source]¶ Generate a JSON file with autotest job summary on a given results directory
Parameters: results_dir – Path to the results directory.
-
autotest.client.shared.report.
get_info_file
(filename)[source]¶ Gets the contents of an autotest info file.
It also and highlights the file contents with possible problems.
Parameters: filename – Info file path.
-
autotest.client.shared.report.
parse_results_dir
(results_dir, relative_links=True)[source]¶ Parse a top level status file and produce a dictionary with job data.
Parameters: dirname – Autotest results directory path Returns: Dictionary with job data.
-
autotest.client.shared.report.
write_html_report
(results_dir, report_path=None, encoding='utf8')[source]¶ Write an HTML file at report_path, with job data summary.
If no report_path specified, generate one at results_dir/job_report.html.
Parameters: - results_dir – Directory with test results.
- report_path – Path to a report file (optional).
- encoding – Encoding for output (optional).
service
Module¶
-
autotest.client.shared.service.
ServiceManager
(run=<function run>)[source]¶ Detect which init program is being used, init or systemd and return a class has methods to start/stop services.
# Get the system service manager service_manager = ServiceManager()
# Stating service/unit “sshd” service_manager.start(“sshd”)
# Getting a list of available units units = service_manager.list()
# Disabling and stopping a list of services services_to_disable = [‘ntpd’, ‘httpd’] for s in services_to_disable:
service_manager.disable(s) service_manager.stop(s)Returns: SysVInitServiceManager or SystemdServiceManager Return type: _GenericServiceManager
-
autotest.client.shared.service.
SpecificServiceManager
(service_name, run=<function run>)[source]¶ # Get the specific service manager for sshd sshd = SpecificServiceManager(“sshd”) sshd.start() sshd.stop() sshd.reload() sshd.restart() sshd.condrestart() sshd.status() sshd.enable() sshd.disable() sshd.is_enabled()
Parameters: service_name (str) – systemd unit or init.d service to manager Returns: SpecificServiceManager that has start/stop methods Return type: _SpecificServiceManager
-
autotest.client.shared.service.
convert_systemd_target_to_runlevel
(target)[source]¶ Convert systemd target to runlevel.
Parameters: target (str) – systemd target Returns: sys_v runlevel Return type: str Raises: ValueError – when systemd target is unknown
-
autotest.client.shared.service.
convert_sysv_runlevel
(level)[source]¶ Convert runlevel to systemd target.
Parameters: level (str or int) – sys_v runlevel Returns: systemd target Return type: str Raises: ValueError – when runlevel is unknown
-
autotest.client.shared.service.
get_name_of_init
(run=<function run>)[source]¶ Determine what executable is PID 1, aka init by checking /proc/1/exe This init detection will only run once and cache the return value.
Returns: executable name for PID 1, aka init Return type: str
-
autotest.client.shared.service.
sys_v_init_command_generator
(command)[source]¶ Generate lists of command arguments for sys_v style inits.
Parameters: command (str) – start,stop,restart, etc. Returns: list of commands to pass to utils.run or similar function Return type: list
-
autotest.client.shared.service.
sys_v_init_result_parser
(command)[source]¶ Parse results from sys_v style commands.
Parameters: command (str.) – command. Returns: different from the command. command is status: return true if service is running. command is is_enabled: return true if service is enalbled. command is list: return a dict from service name to status. command is others: return true if operate success.
-
autotest.client.shared.service.
systemd_command_generator
(command)[source]¶ Generate list of command line argument strings for systemctl. One argument per string for compatibility Popen
WARNING: If systemctl detects that it is running on a tty it will use color, pipe to $PAGER, change column sizes and not truncate unit names. Use –no-pager to suppress pager output, or set PAGER=cat in the environment. You may need to take other steps to suppress color output. See https://bugzilla.redhat.com/show_bug.cgi?id=713567
Parameters: command (str) – start,stop,restart, etc. Returns: list of command and arguments to pass to utils.run or similar functions Return type: list
-
autotest.client.shared.service.
systemd_result_parser
(command)[source]¶ Parse results from systemd style commands.
Parameters: command (str.) – command. Returns: different from the command. command is status: return true if service is running. command is is_enabled: return true if service is enalbled. command is list: return a dict from service name to status. command is others: return true if operate success.
settings
Module¶
A singleton class for accessing global config values.
provides access to global configuration file.
-
class
autotest.client.shared.settings.
Settings
[source]¶ Bases:
object
-
config
= None¶
-
config_file
= '/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/global_config.ini'¶
-
get_section_values
(sections)[source]¶ Return a config parser object containing a single section of the global configuration, that can be later written to a file object.
Parameters: section – Tuple with sections we want to turn into a config parser object. Returns: ConfigParser() object containing all the contents of sections.
-
override_value
(section, key, new_value)[source]¶ Override a value from the config file with a new value.
-
running_stand_alone_client
= False¶
-
set_config_files
(config_file='/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/global_config.ini', shadow_file='/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/shadow_config.ini')[source]¶
-
shadow_file
= '/home/docs/checkouts/readthedocs.org/user_builds/autotest/checkouts/latest/shadow_config.ini'¶
-
software_manager
Module¶
Software package management library.
This is an abstraction layer on top of the existing distributions high level package managers. It supports package operations useful for testing purposes, and multiple high level package managers (here called backends). If you want to make this lib to support your particular package manager/distro, please implement the given backend class.
author: | Higor Vieira Alves (halves@br.ibm.com) |
---|---|
author: | Lucas Meneghel Rodrigues (lmr@redhat.com) |
author: | Ramon de Carvalho Valle (rcvalle@br.ibm.com) |
copyright: | IBM 2008-2009 |
copyright: | Red Hat 2009-2010 |
-
class
autotest.client.shared.software_manager.
AptBackend
[source]¶ Bases:
autotest.client.shared.software_manager.DpkgBackend
Implements the apt backend for software manager.
Set of operations for the apt package manager, commonly found on Debian and Debian based distributions, such as Ubuntu Linux.
-
add_repo
(repo)[source]¶ Add an apt repository.
Parameters: repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ maverick universe’
-
remove_repo
(repo)[source]¶ Remove an apt repository.
Parameters: repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ maverick universe’
-
-
class
autotest.client.shared.software_manager.
BaseBackend
[source]¶ Bases:
object
This class implements all common methods among backends.
-
class
autotest.client.shared.software_manager.
DpkgBackend
[source]¶ Bases:
autotest.client.shared.software_manager.BaseBackend
This class implements operations executed with the dpkg package manager.
dpkg is a lower level package manager, used by higher level managers such as apt and aptitude.
-
INSTALLED_OUTPUT
= 'install ok installed'¶
-
PACKAGE_TYPE
= 'deb'¶
-
-
class
autotest.client.shared.software_manager.
RpmBackend
[source]¶ Bases:
autotest.client.shared.software_manager.BaseBackend
This class implements operations executed with the rpm package manager.
rpm is a lower level package manager, used by higher level managers such as yum and zypper.
-
PACKAGE_TYPE
= 'rpm'¶
-
SOFTWARE_COMPONENT_QRY
= 'rpm %{NAME} %{VERSION} %{RELEASE} %{SIGMD5} %{ARCH}'¶
-
check_installed
(name, version=None, arch=None)[source]¶ Check if package [name] is installed.
Parameters: - name – Package name.
- version – Package version.
- arch – Package architecture.
-
-
class
autotest.client.shared.software_manager.
SoftwareManager
[source]¶ Bases:
object
Package management abstraction layer.
It supports a set of common package operations for testing purposes, and it uses the concept of a backend, a helper class that implements the set of operations of a given package management tool.
-
class
autotest.client.shared.software_manager.
SoftwareManagerLoggingConfig
(use_console=True)[source]¶ Bases:
autotest.client.shared.logging_config.LoggingConfig
Used with the sole purpose of providing logging setup for this program.
-
class
autotest.client.shared.software_manager.
SystemInspector
[source]¶ Bases:
object
System inspector class.
This may grow up to include more complete reports of operating system and machine properties.
-
class
autotest.client.shared.software_manager.
YumBackend
[source]¶ Bases:
autotest.client.shared.software_manager.RpmBackend
Implements the yum backend for software manager.
Set of operations for the yum package manager, commonly found on Yellow Dog Linux and Red Hat based distributions, such as Fedora and Red Hat Enterprise Linux.
-
add_repo
(url)[source]¶ Adds package repository located on [url].
Parameters: url – Universal Resource Locator of the repository.
-
provides
(name)[source]¶ Returns a list of packages that provides a given capability.
Parameters: name – Capability name (eg, ‘foo’).
-
-
class
autotest.client.shared.software_manager.
ZypperBackend
[source]¶ Bases:
autotest.client.shared.software_manager.RpmBackend
Implements the zypper backend for software manager.
Set of operations for the zypper package manager, found on SUSE Linux.
-
install
(name)[source]¶ Installs package [name]. Handles local installs.
Parameters: name – Package Name.
-
-
autotest.client.shared.software_manager.
install_distro_packages
(distro_pkg_map, interactive=False)[source]¶ Installs packages for the currently running distribution
This utility function checks if the currently running distro is a key in the distro_pkg_map dictionary, and if there is a list of packages set as its value.
If these conditions match, the packages will be installed using the software manager interface, thus the native packaging system if the currenlty running distro.
Parameters: distro_pkg_map (dict) – mapping of distro name, as returned by utils.get_os_vendor(), to a list of package names Returns: True if any packages were actually installed, False otherwise
ssh_key
Module¶
syncdata
Module¶
test
Module¶
-
class
autotest.client.shared.test.
Subtest
[source]¶ Bases:
object
Collect result of subtest of main test.
-
clean
()[source]¶ Check if cleanup is defined.
For makes test fatal add before implementation of test method decorator @subtest_nocleanup
-
failed
= 0¶
-
classmethod
get_result
()[source]¶ Returns: Result of subtests. Format: tuple(pass/fail,function_name,call_arguments)
-
classmethod
log_append
(msg)[source]¶ Add log_append to result output.
Parameters: msg – Test of log_append
-
passed
= 0¶
-
result
= []¶
-
static
result_to_string
(result)[source]¶ Format of result dict.
- result = {
- ‘result’ : “PASS” / “FAIL”, ‘name’ : class name, ‘args’ : test’s args, ‘kargs’ : test’s kargs, ‘output’ : return of test function,
}
Parameters: result – Result of test.
-
runsubtest
(url, *args, **dargs)[source]¶ Execute another autotest test from inside the current test’s scope.
Parameters: - test – Parent test.
- url – Url of new test.
- tag – Tag added to test name.
- args – Args for subtest.
- dargs – Dictionary with args for subtest.
@iterations: Number of subtest iterations. @profile_only: If true execute one profiled run.
-
-
class
autotest.client.shared.test.
base_test
(job, bindir, outputdir)[source]¶ Bases:
object
-
after_run_once
()[source]¶ Called after every run_once (including from a profiled run when it’s called after stopping the profilers).
-
before_run_once
()[source]¶ Override in tests that need it, will be called before any run_once() call including the profiling run (when it’s called before starting the profilers).
-
execute
(iterations=None, test_length=None, profile_only=None, _get_time=<built-in function time>, postprocess_profiled_run=None, constraints=(), *args, **dargs)[source]¶ This is the basic execute method for the tests inherited from base_test. If you want to implement a benchmark test, it’s better to implement the run_once function, to cope with the profiling infrastructure. For other tests, you can just override the default implementation.
Parameters: - test_length – The minimum test length in seconds. We’ll run the run_once function for a number of times large enough to cover the minimum test length.
- iterations – A number of iterations that we’ll run the run_once function. This parameter is incompatible with test_length and will be silently ignored if you specify both.
- profile_only – If true run X iterations with profilers enabled. If false run X iterations and one with profiling if profiles are enabled. If None, default to the value of job.default_profile_only.
- _get_time – [time.time] Used for unit test time injection.
- postprocess_profiled_run – Run the postprocessing for the profiled run.
-
network_destabilizing
= False¶
-
preserve_srcdir
= False¶
-
register_after_iteration_hook
(iteration_hook)[source]¶ This is how we expect test writers to register an after_iteration_hook. This adds the method to the list of hooks which are executed after each iteration.
Parameters: iteration_hook – Method to run after each iteration. A valid hook accepts a single argument which is the test object.
-
register_before_iteration_hook
(iteration_hook)[source]¶ This is how we expect test writers to register a before_iteration_hook. This adds the method to the list of hooks which are executed before each iteration.
Parameters: iteration_hook – Method to run before each iteration. A valid hook accepts a single argument which is the test object.
-
-
autotest.client.shared.test.
runtest
(job, url, tag, args, dargs, local_namespace={}, global_namespace={}, before_test_hook=None, after_test_hook=None, before_iteration_hook=None, after_iteration_hook=None)[source]¶
utils
Module¶
Convenience functions for use by tests or whomever.
NOTE: this is a mixin library that pulls in functions from several places Note carefully what the precendece order is
There’s no really good way to do this, as this isn’t a class we can do inheritance with, just a collection of static methods.
-
class
autotest.client.shared.utils.
AsyncJob
(command, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_level=40, kill_func=None, close_fds=False)[source]¶
-
class
autotest.client.shared.utils.
BgJob
(command, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_level=40, close_fds=False)[source]¶ Bases:
object
-
class
autotest.client.shared.utils.
CmdResult
(command='', stdout='', stderr='', exit_status=None, duration=0)[source]¶ Bases:
object
Command execution result.
command: String containing the command line itself exit_status: Integer exit code of the process stdout: String containing stdout of the process stderr: String containing stderr of the process duration: Elapsed wall clock time running the process
-
class
autotest.client.shared.utils.
FileFieldMonitor
(status_file, data_to_read, mode_diff, continuously=False, contlogging=False, separator=' +', time_step=0.1)[source]¶ Bases:
object
Monitors the information from the file and reports it’s values.
It gather the information at start and stop of the measurement or continuously during the measurement.
-
class
Monitor
(master)[source]¶ Bases:
threading.Thread
Internal monitor class to ensure continuous monitor of monitored file.
-
class
-
class
autotest.client.shared.utils.
ForAllPSE
[source]¶ Bases:
list
Parallel version of and suppress exception.
-
class
autotest.client.shared.utils.
InterruptedThread
(target, args=(), kwargs={})[source]¶ Bases:
threading.Thread
Run a function in a background thread.
-
class
autotest.client.shared.utils.
Statistic
[source]¶ Bases:
object
Class to display and collect average, max and min values of a given data set.
-
class
autotest.client.shared.utils.
SystemLoad
(pids, advanced=False, time_step=0.1, cpu_cont=False, use_log=False)[source]¶ Bases:
object
Get system and/or process values and return average value of load.
-
dump
(pids=[])[source]¶ Get the status of monitoring. :param pids: List of PIDs you intend to control. Use pids=[] to control
all defined PIDs.return: - tuple([cpu load], [memory load]):
- ([(PID1, (PID1_cpu_meas)), (PID2, (PID2_cpu_meas)), …],
[(PID1, (PID1_mem_meas)), (PID2, (PID2_mem_meas)), …])
- PID1_cpu_meas:
average_values[], test_time, cont_meas_values[[]], time_step
- PID1_mem_meas:
average_values[], test_time, cont_meas_values[[]], time_step
where average_values[] are the measured values (mem_free,swap,…) which are described in SystemLoad.__init__()-FileFieldMonitor. cont_meas_values[[]] is a list of average_values in the sampling times.
-
get_cpu_status_string
(pids=[])[source]¶ Convert status to string array. :param pids: List of PIDs you intend to control. Use pids=[] to control
all defined PIDs.Returns: String format to table.
-
get_mem_status_string
(pids=[])[source]¶ Convert status to string array. :param pids: List of PIDs you intend to control. Use pids=[] to control
all defined PIDs.Returns: String format to table.
-
-
class
autotest.client.shared.utils.
VersionableClass
[source]¶ Bases:
object
VersionableClass provides class hierarchy which automatically select right version of class. Class manipulation is used for this reason. By this reason is: Advantage) Only one version is working in one process. Class is changed in whole process. Disadvantage) Only one version is working in one process.
Example of usage (in utils_unittest):
class FooC(object): pass #Not implemented get_version -> not used for versioning. class VCP(FooC, VersionableClass): def __new__(cls, *args, **kargs): VCP.master_class = VCP return super(VCP, cls).__new__(cls, *args, **kargs) def foo(self): pass class VC2(VCP, VersionableClass): @staticmethod def get_version(): return "get_version_from_system" @classmethod def is_right_version(cls, version): if version is not None: if "version is satisfied": return True return False def func1(self): print "func1" def func2(self): print "func2" # get_version could be inherited. class VC3(VC2, VersionableClass): @classmethod def is_right_version(cls, version): if version is not None: if "version+1 is satisfied": return True return False def func2(self): print "func2_2" class M(VCP): pass m = M() # <- When class is constructed the right version is # automatically selected. In this case VC3 is selected. m.func2() # call VC3.func2(m) m.func1() # call VC2.func1(m) m.foo() # call VC1.foo(m) # When controlled "program" version is changed then is necessary call check_repair_versions or recreate object. m.check_repair_versions() # priority of class. (change place where is method searched first in group # of verisonable class.) class PP(VersionableClass): def __new__(cls, *args, **kargs): PP.master_class = PP return super(PP, cls).__new__(cls, *args, **kargs) class PP2(PP, VersionableClass): @staticmethod def get_version(): return "get_version_from_system" @classmethod def is_right_version(cls, version): if version is not None: if "version is satisfied": return True return False def func1(self): print "PP func1" class N(VCP, PP): pass n = N() n.func1() # -> "func2" n.set_priority_class(PP, [VCP, PP]) n.func1() # -> "PP func1"
Necessary for using: 1) Subclass of versionable class must have implemented class methods get_version and is_right_version. These two methods are necessary for correct version section. Class without this method will be never chosen like suitable class.
2) Every class derived from master_class have to add to class definition inheritance from VersionableClass. Direct inheritance from Versionable Class is use like a mark for manipulation with VersionableClass.
3) Master of VersionableClass have to defined class variable cls.master_class.
-
classmethod
check_repair_versions
(master_classes=None)[source]¶ Check version of versionable class and if version not match repair version to correct version.
Parameters: master_classes (list.) – Check and repair only master_class.
-
classmethod
get_version
()[source]¶ Get version of installed OpenVSwtich. Must be re-implemented for in child class.
Returns: Version or None when get_version is unsuccessful.
-
classmethod
-
autotest.client.shared.utils.
archive_as_tarball
(source_dir, dest_dir, tarball_name=None, compression='bz2', verbose=True)[source]¶ Saves the given source directory to the given destination as a tarball
If the name of the archive is omitted, it will be taken from the source_dir. If it is an absolute path, dest_dir will be ignored. But, if both the destination directory and tarball anem is given, and the latter is not an absolute path, they will be combined.
For archiving directory ‘/tmp’ in ‘/net/server/backup’ as file ‘tmp.tar.bz2’, simply use:
>>> utils.archive_as_tarball('/tmp', '/net/server/backup')
To save the file it with a different name, say ‘host1-tmp.tar.bz2’ and save it under ‘/net/server/backup’, use:
>>> utils.archive_as_tarball('/tmp', '/net/server/backup', 'host1-tmp')
To save with gzip compression instead (resulting in the file ‘/net/server/backup/host1-tmp.tar.gz’), use:
>>> utils.archive_as_tarball('/tmp', '/net/server/backup', 'host1-tmp', 'gz')
-
autotest.client.shared.utils.
args_to_dict
(args)[source]¶ Convert autoserv extra arguments in the form of key=val or key:val to a dictionary. Each argument key is converted to lowercase dictionary key.
- Args:
- args - list of autoserv extra arguments.
- Returns:
- dictionary
-
autotest.client.shared.utils.
ask
(question, auto=False)[source]¶ Raw input with a prompt that emulates logging.
Parameters: - question – Question to be asked
- auto – Whether to return “y” instead of asking the question
-
autotest.client.shared.utils.
aton
(sr)[source]¶ Transform a string to a number(include float and int). If the string is not in the form of number, just return false.
Parameters: sr – string to transfrom Returns: float, int or False for failed transform
-
autotest.client.shared.utils.
bitlist_to_string
(data)[source]¶ Transform from bit list to ASCII string.
Parameters: data – Bit list to be transformed
-
autotest.client.shared.utils.
compare_versions
(ver1, ver2)[source]¶ Version number comparison between ver1 and ver2 strings.
>>> compare_tuple("1", "2") -1 >>> compare_tuple("foo-1.1", "foo-1.2") -1 >>> compare_tuple("1.2", "1.2a") -1 >>> compare_tuple("1.2b", "1.2a") 1 >>> compare_tuple("1.3.5.3a", "1.3.5.3b") -1
- Args:
- ver1: version string ver2: version string
- Returns:
- int: 1 if ver1 > ver2
- 0 if ver1 == ver2
-1 if ver1 < ver2
-
autotest.client.shared.utils.
configure
(extra=None, configure='./configure')[source]¶ Run configure passing in the correct host, build, and target options.
Parameters: - extra – extra command line arguments to pass to configure
- configure – which configure script to use
-
autotest.client.shared.utils.
convert_data_size
(size, default_sufix='B')[source]¶ Convert data size from human readable units to an int of arbitrary size.
Parameters: - size – Human readable data size representation (string).
- default_sufix – Default sufix used to represent data.
Returns: Int with data size in the appropriate order of magnitude.
-
autotest.client.shared.utils.
convert_ipv4_to_ipv6
(ipv4)[source]¶ Translates a passed in string of an ipv4 address to an ipv6 address.
Parameters: ipv4 – a string of an ipv4 address
-
autotest.client.shared.utils.
cpu_affinity_by_task
(pid, vcpu_pid)[source]¶ This function returns the allowed cpus from the proc entry for each vcpu’s through its task id for a pid(of a VM)
-
autotest.client.shared.utils.
create_x509_dir
(path, cacert_subj, server_subj, passphrase, secure=False, bits=1024, days=1095)[source]¶ Creates directory with freshly generated: ca-cart.pem, ca-key.pem, server-cert.pem, server-key.pem,
Parameters: - path – defines path to directory which will be created
- cacert_subj – ca-cert.pem subject
:param server_key.csr subject :param passphrase - passphrase to ca-key.pem :param secure = False - defines if the server-key.pem will use a passphrase :param bits = 1024: bit length of keys :param days = 1095: cert expiration
Raises: - ValueError – openssl not found or rc != 0
- OSError – if os.makedirs() fails
-
autotest.client.shared.utils.
delete_pid_file_if_exists
(program_name, pid_files_dir=None)[source]¶ Tries to remove <program_name>.pid from the main autotest directory.
-
autotest.client.shared.utils.
deprecated
(func)[source]¶ This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emitted when the function is used.
-
autotest.client.shared.utils.
display_data_size
(size)[source]¶ Display data size in human readable units.
Parameters: size (int) – Data size, in Bytes. Returns: Human readable string with data size.
-
autotest.client.shared.utils.
etraceback
(prep, exc_info)[source]¶ - Enhanced Traceback formats traceback into lines “prep: line
- name: line”
param prep: desired line preposition param exc_info: sys.exc_info of the exception return: string which contains beautifully formatted exception
-
autotest.client.shared.utils.
find_command
(cmd)[source]¶ Try to find a command in the PATH, paranoid version.
Parameters: cmd – Command to be found. Raise: ValueError in case the command was not found.
-
autotest.client.shared.utils.
find_free_port
(start_port, end_port, address='localhost')[source]¶ Return a host free port in the range [start_port, end_port].
Parameters: - start_port – First port that will be checked.
- end_port – Port immediately after the last one that will be checked.
-
autotest.client.shared.utils.
find_free_ports
(start_port, end_port, count, address='localhost')[source]¶ Return count of host free ports in the range [start_port, end_port].
@count: Initial number of ports known to be free in the range. :param start_port: First port that will be checked. :param end_port: Port immediately after the last one that will be checked.
-
autotest.client.shared.utils.
find_substring
(string, pattern1, pattern2=None)[source]¶ Return the match of pattern1 in string. Or return the match of pattern2 if pattern is not matched.
@string: string @pattern1: first pattern want to match in string, must set. @pattern2: second pattern, it will be used if pattern1 not match, optional.
Return: Match substing or None
-
autotest.client.shared.utils.
format_str_for_message
(msg_str)[source]¶ Format msg_str so that it can be appended to a message. If msg_str consists of one line, prefix it with a space. If msg_str consists of multiple lines, prefix it with a newline.
Parameters: msg_str – string that will be formatted.
-
autotest.client.shared.utils.
generate_random_id
()[source]¶ Return a random string suitable for use as a qemu id.
-
autotest.client.shared.utils.
generate_random_string
(length, ignore_str='!"#$%&\'()*+, -./:;<=>?@[\\]^_`{|}~', convert_str='')[source]¶ Return a random string using alphanumeric characters.
Parameters: - length – Length of the string that will be generated.
- ignore_str – Characters that will not include in generated string.
- convert_str – Characters that need to be escaped (prepend “”).
Returns: The generated random string.
-
autotest.client.shared.utils.
generate_tmp_file_name
(file_name, ext=None, directory='/tmp/')[source]¶ Returns a temporary file name. The file is not created.
-
autotest.client.shared.utils.
get_arch
(run_function=<function run>)[source]¶ Get the hardware architecture of the machine. run_function is used to execute the commands. It defaults to utils.run() but a custom method (if provided) should be of the same schema as utils.run. It should return a CmdResult object and throw a CmdError exception.
-
autotest.client.shared.utils.
get_archive_tarball_name
(source_dir, tarball_name, compression)[source]¶ Get the name for a tarball file, based on source, name and compression
-
autotest.client.shared.utils.
get_children_pids
(ppid)[source]¶ Get all PIDs of children/threads of parent ppid param ppid: parent PID return: list of PIDs of all children/threads of ppid
-
autotest.client.shared.utils.
get_cpu_percentage
(function, *args, **dargs)[source]¶ Returns a tuple containing the CPU% and return value from function call.
This function calculates the usage time by taking the difference of the user and system times both before and after the function call.
-
autotest.client.shared.utils.
get_field
(data, param, linestart='', sep=' ')[source]¶ Parse data from string. :param data: Data to parse.
- example:
- data:
- cpu 324 345 34 5 345 cpu0 34 11 34 34 33 ^^^^ start of line params 0 1 2 3 4
Parameters: - param – Position of parameter after linestart marker.
- linestart – String to which start line with parameters.
- sep – Separator between parameters regular expression.
-
autotest.client.shared.utils.
get_file
(src, dest, permissions=None)[source]¶ Get a file from src, which can be local or a remote URL
-
autotest.client.shared.utils.
get_full_pci_id
(pci_id)[source]¶ Get full PCI ID of pci_id.
Parameters: pci_id – PCI ID of a device.
-
autotest.client.shared.utils.
get_hash_from_file
(hash_path, dvd_basename)[source]¶ Get the a hash from a given DVD image from a hash file (Hash files are usually named MD5SUM or SHA1SUM and are located inside the download directories of the DVDs)
Parameters: - hash_path – Local path to a hash file.
- cd_image – Basename of a CD image
-
autotest.client.shared.utils.
get_num_logical_cpus_per_socket
(run_function=<function run>)[source]¶ Get the number of cores (including hyperthreading) per cpu. run_function is used to execute the commands. It defaults to utils.run() but a custom method (if provided) should be of the same schema as utils.run. It should return a CmdResult object and throw a CmdError exception.
-
autotest.client.shared.utils.
get_path
(base_path, user_path)[source]¶ Translate a user specified path to a real path. If user_path is relative, append it to base_path. If user_path is absolute, return it as is.
Parameters: - base_path – The base path of relative user specified paths.
- user_path – The user specified path.
-
autotest.client.shared.utils.
get_pid_cpu
(pid)[source]¶ Get the process used cpus.
Parameters: pid – process id Returns: A list include all cpus the process used Return type: list
-
autotest.client.shared.utils.
get_pid_from_file
(program_name, pid_files_dir=None)[source]¶ Reads the pid from <program_name>.pid in the autotest directory.
:param program_name the name of the program :return: the pid if the file exists, None otherwise.
-
autotest.client.shared.utils.
get_process_name
(pid)[source]¶ Get process name from PID. :param pid: PID of process.
-
autotest.client.shared.utils.
get_relative_path
(path, reference)[source]¶ Given 2 absolute paths “path” and “reference”, compute the path of “path” as relative to the directory “reference”.
:param path the absolute path to convert to a relative path :param reference an absolute directory path to which the relative
path will be computed
-
autotest.client.shared.utils.
get_thread_cpu
(thread)[source]¶ Get the light weight process(thread) used cpus.
Parameters: thread (string) – thread checked Returns: A list include all cpus the thread used Return type: list
-
autotest.client.shared.utils.
get_unique_name
(check, prefix='', suffix='', length=None, skip=None)[source]¶ Get unique name according to check function, use only 1000 iterations. :param cmp: Function called to discover name uniqueness :param prefix: Name prefix :param suffix: Name suffix :param length: Length of random string, when None use numbers (0,1,2) :param skip: skip n numbers (only when length=None
Raises: StopIteration – In case no unique name obtained in 1000 iterations Returns: Unique name according to check function
-
autotest.client.shared.utils.
get_unused_port
()[source]¶ Finds a semi-random available port. A race condition is still possible after the port number is returned, if another process happens to bind it.
- Returns:
- A port number that is unused on both TCP and UDP.
-
autotest.client.shared.utils.
get_vendor_from_pci_id
(pci_id)[source]¶ Check out the device vendor ID according to pci_id.
Parameters: pci_id – PCI ID of a device.
-
autotest.client.shared.utils.
hash
(type, input=None)[source]¶ Returns an hash object of type md5 or sha1. This function is implemented in order to encapsulate hash objects in a way that is compatible with python 2.4 and python 2.6 without warnings.
Note that even though python 2.6 hashlib supports hash types other than md5 and sha1, we are artificially limiting the input values in order to make the function to behave exactly the same among both python implementations.
Parameters: input – Optional input string that will be used to update the hash.
-
autotest.client.shared.utils.
import_site_class
(path, module, classname, baseclass, modulefile=None)[source]¶ Try to import site specific class from site specific file if it exists
- Args:
path: full filename of the source file calling this (ie __file__) module: full module name classname: class name to be loaded from site file baseclass: base class object to return when no site file present or
to mixin when site class exists but is not inherited from baseclassmodulefile: module filename
- Returns: baseclass if site specific class does not exist, the site specific
- class if it exists and is inherited from baseclass or a mixin of the site specific class and baseclass when the site specific class exists and is not inherited from baseclass
Raises: ImportError if the site file exists but imports fails
-
autotest.client.shared.utils.
import_site_function
(path, module, funcname, dummy, modulefile=None)[source]¶ Try to import site specific function from site specific file if it exists
- Args:
- path: full filename of the source file calling this (ie __file__) module: full module name funcname: function name to be imported from site file dummy: dummy function to return in case there is no function to import modulefile: module filename
Returns: site specific function object or dummy
Raises: ImportError if the site file exists but imports fails
-
autotest.client.shared.utils.
import_site_module
(path, module, dummy=None, modulefile=None)[source]¶ Try to import the site specific module if it exists.
:param path full filename of the source file calling this (ie __file__) :param module full module name :param dummy dummy value to return in case there is no symbol to import :param modulefile module filename
Returns: site specific module or dummy :raise ImportError if the site file exists but imports fails
-
autotest.client.shared.utils.
import_site_symbol
(path, module, name, dummy=None, modulefile=None)[source]¶ Try to import site specific symbol from site specific file if it exists
:param path full filename of the source file calling this (ie __file__) :param module full module name :param name symbol name to be imported from the site file :param dummy dummy value to return in case there is no symbol to import :param modulefile module filename
Returns: site specific symbol or dummy :raise ImportError if the site file exists but imports fails
-
autotest.client.shared.utils.
interactive_download
(url, output_file, title='', chunk_size=102400)[source]¶ Interactively downloads a given file url to a given output file
Parameters: - url (string) – URL for the file to be download
- output_file (string) – file name or absolute path on which to save the file to
- title (string) – optional title to go along the progress bar
- chunk_size (integer) – amount of data to read at a time
-
autotest.client.shared.utils.
is_mounted
(src, mount_point, fstype, perm=None, verbose=True, fstype_mtab=None)[source]¶ Check mount status from /etc/mtab
Parameters: - src (string) – mount source
- mount_point (string) – mount point
- fstype (string) – file system type
- perm (string) – mount permission
- fstype_mtab (str) – file system type in mtab could be different
Returns: if the src is mounted as expect
Return type: Boolean
-
autotest.client.shared.utils.
is_port_free
(port, address)[source]¶ Return True if the given port is available for use.
Parameters: port – Port number
-
autotest.client.shared.utils.
join_bg_jobs
(bg_jobs, timeout=None)[source]¶ Joins the bg_jobs with the current thread.
Returns the same list of bg_jobs objects that was passed in.
-
autotest.client.shared.utils.
kill_process_tree
(pid, sig=9)[source]¶ Signal a process and all of its children.
If the process does not exist – return.
Parameters: - pid – The pid of the process to signal.
- sig – The signal to send to the processes.
-
autotest.client.shared.utils.
log_last_traceback
(msg=None, log=<function error>)[source]¶ Writes last traceback into specified log. :param msg: Override the default message. [“Original traceback”] :param log: Where to log the traceback [logging.error]
-
autotest.client.shared.utils.
log_line
(filename, line)[source]¶ - Write a line to a file. ‘
‘ is appended to the line.
param filename: Path of file to write to, either absolute or relative to the dir set by set_log_file_dir(). param line: Line to write.
-
autotest.client.shared.utils.
make
(extra='', make='make', timeout=None, ignore_status=False)[source]¶ Run make, adding MAKEOPTS to the list of options.
Parameters: extra – extra command line arguments to pass to make.
-
autotest.client.shared.utils.
matrix_to_string
(matrix, header=None)[source]¶ Return a pretty, aligned string representation of a nxm matrix.
This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.
Parameters: - matrix – Matrix representation (list with n rows of m elements).
- header – Optional tuple or list with header elements to be displayed.
-
autotest.client.shared.utils.
merge_trees
(src, dest)[source]¶ Merges a source directory tree at ‘src’ into a destination tree at ‘dest’. If a path is a file in both trees than the file in the source tree is APPENDED to the one in the destination tree. If a path is a directory in both trees then the directories are recursively merged with this function. In any other case, the function will skip the paths that cannot be merged (instead of failing).
-
autotest.client.shared.utils.
mount
(src, mount_point, fstype, perm=None, verbose=True, fstype_mtab=None)[source]¶ Mount the src into mount_point of the host.
Src: mount source Mount_point: mount point Fstype: file system type Perm: mount permission Parameters: fstype_mtab (str) – file system type in mtab could be different
-
autotest.client.shared.utils.
parallel
(targets)[source]¶ Run multiple functions in parallel.
Parameters: targets – A sequence of tuples or functions. If it’s a sequence of tuples, each tuple will be interpreted as (target, args, kwargs) or (target, args) or (target,) depending on its length. If it’s a sequence of functions, the functions will be called without arguments. Returns: A list of the values returned by the functions called.
-
autotest.client.shared.utils.
pid_exists
(pid)[source]¶ Return True if a given PID exists.
Parameters: pid – Process ID number.
-
autotest.client.shared.utils.
pid_is_alive
(pid)[source]¶ True if process pid exists and is not yet stuck in Zombie state. Zombies are impossible to move between cgroups, etc. pid can be integer, or text of integer.
-
autotest.client.shared.utils.
process_or_children_is_defunct
(ppid)[source]¶ Verify if any processes from PPID is defunct.
Attempt to verify if parent process and any children from PPID is defunct (zombie) or not. :param ppid: The parent PID of the process to verify.
-
autotest.client.shared.utils.
program_is_alive
(program_name, pid_files_dir=None)[source]¶ Checks if the process is alive and not in Zombie state.
:param program_name the name of the program :return: True if still alive, False otherwise
-
autotest.client.shared.utils.
read_keyval
(path)[source]¶ Read a key-value pair format file into a dictionary, and return it. Takes either a filename or directory name as input. If it’s a directory name, we assume you want the file to be called keyval.
-
autotest.client.shared.utils.
run
(command, timeout=None, ignore_status=False, stdout_tee=None, stderr_tee=None, verbose=True, stdin=None, stderr_is_expected=None, args=())[source]¶ Run a command on the host.
Parameters: - command – the command line string.
- timeout – time limit in seconds before attempting to kill the running process. The run() function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process.
- ignore_status – do not raise an exception, no matter what the exit code of the command is.
- stdout_tee – optional file-like object to which stdout data will be written as it is generated (data will still be stored in result.stdout).
- stderr_tee – likewise for stderr.
- verbose – if True, log the command being run.
- stdin – stdin to pass to the executed process (can be a file descriptor, a file object of a real file or a string).
- args – sequence of strings of arguments to be given to the command inside ” quotes after they have been escaped for that; each element in the sequence will be given as a separate command argument
Returns: a CmdResult object
Raises: CmdError – the exit code of the command execution was not 0
-
autotest.client.shared.utils.
run_bg
(*args, **dargs)[source]¶ Function deprecated. Please use BgJob class instead.
-
autotest.client.shared.utils.
run_parallel
(commands, timeout=None, ignore_status=False, stdout_tee=None, stderr_tee=None)[source]¶ Behaves the same as run() with the following exceptions:
- commands is a list of commands to run in parallel.
- ignore_status toggles whether or not an exception should be raised on any error.
Returns: a list of CmdResult objects
-
autotest.client.shared.utils.
safe_kill
(pid, signal)[source]¶ Attempt to send a signal to a given process that may or may not exist.
Parameters: signal – Signal number.
-
autotest.client.shared.utils.
safe_rmdir
(path, timeout=10)[source]¶ Try to remove a directory safely, even on NFS filesystems.
Sometimes, when running an autotest client test on an NFS filesystem, when not all filedescriptors are closed, NFS will create some temporary files, that will make shutil.rmtree to fail with error 39 (directory not empty). So let’s keep trying for a reasonable amount of time before giving up.
Parameters: - path (string) – Path to a directory to be removed.
- timeout (int) – Time that the function will try to remove the dir before giving up (seconds)
Raises: OSError, with errno 39 in case after the timeout shutil.rmtree could not successfuly complete. If any attempt to rmtree fails with errno different than 39, that exception will be just raised.
-
autotest.client.shared.utils.
selinux_enforcing
()[source]¶ Returns True if SELinux is in enforcing mode, False if permissive/disabled
-
autotest.client.shared.utils.
set_log_file_dir
(directory)[source]¶ Set the base directory for log files created by log_line().
Parameters: dir – Directory for log files.
-
autotest.client.shared.utils.
sh_escape
(command)[source]¶ Escape special characters from a command so that it can be passed as a double quoted (” “) string in a (ba)sh command.
- Args:
- command: the command string to escape.
- Returns:
- The escaped command string. The required englobing double quotes are NOT added and so should be added at some point by the caller.
See also: http://www.tldp.org/LDP/abs/html/escapingsection.html
-
autotest.client.shared.utils.
signal_pid
(pid, sig)[source]¶ Sends a signal to a process id. Returns True if the process terminated successfully, False otherwise.
-
autotest.client.shared.utils.
signal_program
(program_name, sig=15, pid_files_dir=None)[source]¶ Sends a signal to the process listed in <program_name>.pid
:param program_name the name of the program :param sig signal to send
-
autotest.client.shared.utils.
string_to_bitlist
(data)[source]¶ Transform from ASCII string to bit list.
Parameters: data – String to be transformed
-
autotest.client.shared.utils.
strip_console_codes
(output)[source]¶ Remove the Linux console escape and control sequences from the console output. Make the output readable and can be used for result check. Now only remove some basic console codes using during boot up.
Parameters: output (string) – The output from Linux console Returns: the string wihout any special codes Return type: string
-
autotest.client.shared.utils.
system
(command, timeout=None, ignore_status=False, verbose=True)[source]¶ Run a command
Parameters: - timeout – timeout in seconds
- ignore_status – if ignore_status=False, throw an exception if the command’s exit code is non-zero if ignore_status=True, return the exit code.
- verbose – if True, log the command being run.
Returns: exit status of command (note, this will always be zero unless ignore_status=True)
-
autotest.client.shared.utils.
system_output
(command, timeout=None, ignore_status=False, retain_output=False, args=(), verbose=True)[source]¶ Run a command and return the stdout output.
Parameters: - command – command string to execute.
- timeout – time limit in seconds before attempting to kill the running process. The function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process.
- ignore_status – do not raise an exception, no matter what the exit code of the command is.
- retain_output – set to True to make stdout/stderr of the command output to be also sent to the logging system
- args – sequence of strings of arguments to be given to the command inside ” quotes after they have been escaped for that; each element in the sequence will be given as a separate command argument
- verbose – if True, log the command being run.
Returns: a string with the stdout output of the command.
-
autotest.client.shared.utils.
system_output_parallel
(commands, timeout=None, ignore_status=False, retain_output=False)[source]¶
-
autotest.client.shared.utils.
system_parallel
(commands, timeout=None, ignore_status=False)[source]¶ This function returns a list of exit statuses for the respective list of commands.
-
autotest.client.shared.utils.
umount
(src, mount_point, fstype, verbose=True, fstype_mtab=None)[source]¶ Umount the src mounted in mount_point.
Src: mount source Mount_point: mount point Type: file system type Parameters: fstype_mtab (str) – file system type in mtab could be different
-
autotest.client.shared.utils.
unique
(llist)[source]¶ Return a list of the elements in list, but without duplicates.
Parameters: list – List with values. Returns: List with non duplicate elements.
-
autotest.client.shared.utils.
unmap_url
(srcdir, src, destdir='.')[source]¶ Receives either a path to a local file or a URL. returns either the path to the local file, or the fetched URL
- unmap_url(‘/usr/src’, ‘foo.tar’, ‘/tmp’)
- = ‘/usr/src/foo.tar’
- unmap_url(‘/usr/src’, ‘http://site/file’, ‘/tmp’)
- = ‘/tmp/file’ (after retrieving it)
-
autotest.client.shared.utils.
update_version
(srcdir, preserve_srcdir, new_version, install, *args, **dargs)[source]¶ Make sure srcdir is version new_version
If not, delete it and install() the new version.
In the preserve_srcdir case, we just check it’s up to date, and if not, we rerun install, without removing srcdir
-
autotest.client.shared.utils.
urlopen
(url, data=None, timeout=5)[source]¶ Wrapper to urllib2.urlopen with timeout addition.
-
autotest.client.shared.utils.
urlretrieve
(url, filename, data=None, timeout=300)[source]¶ Retrieve a file from given url.
-
autotest.client.shared.utils.
verify_running_as_root
()[source]¶ Verifies whether we’re running under UID 0 (root).
Raise: error.TestNAError
-
autotest.client.shared.utils.
wait_for
(func, timeout, first=0.0, step=1.0, text=None)[source]¶ If func() evaluates to True before timeout expires, return the value of func(). Otherwise return None.
@brief: Wait until func() evaluates to True.
Parameters: - timeout – Timeout in seconds
- first – Time to sleep before first attempt
- steps – Time to sleep between attempts in seconds
- text – Text to print while waiting, for debug purposes
-
autotest.client.shared.utils.
write_keyval
(path, dictionary, type_tag=None, tap_report=None)[source]¶ Write a key-value pair format file out to a file. This uses append mode to open the file, so existing text will not be overwritten or reparsed.
If type_tag is None, then the key must be composed of alphanumeric characters (or dashes+underscores). However, if type-tag is not null then the keys must also have “{type_tag}” as a suffix. At the moment the only valid values of type_tag are “attr” and “perf”.
Parameters: - path – full path of the file to be written
- dictionary – the items to write
- type_tag – see text above
utils_cgroup
Module¶
Helpers for cgroup testing.
copyright: | 2011 Red Hat Inc. |
---|---|
author: | Lukas Doktor <ldoktor@redhat.com> |
-
class
autotest.client.shared.utils_cgroup.
Cgroup
(module, _client)[source]¶ Bases:
object
Cgroup handling class.
-
cgclassify_cgroup
(pid, cgroup)[source]¶ Classify pid into cgroup
Parameters: - pid – pid of the process
- cgroup – cgroup name
-
cgdelete_cgroup
(cgroup, recursive=False)[source]¶ Delete desired cgroup.
Params cgroup: desired cgroup :params force:If true, sub cgroup can be deleted with parent cgroup
-
cgexec
(cgroup, cmd, args='')[source]¶ Execute command in desired cgroup
Param: cgroup: Desired cgroup Param: cmd: Executed command Param: args: Executed command’s parameters
-
cgset_property
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the property value by cgset command
Param: prop: property name (file)
Param: value: desired value
Parameters: - pwd – cgroup directory
- check – check the value after setup / override checking value
- checkprop – override prop when checking the value
-
get_cgroup_index
(cgroup)[source]¶ Get cgroup’s index in cgroups
Param: cgroup: cgroup name Returns: index of cgroup
-
get_pids
(pwd=None)[source]¶ Get all pids in cgroup
Params: pwd: cgroup directory Returns: all pids(list)
-
get_property
(prop, pwd=None)[source]¶ Gets the property value :param prop: property name (file) :param pwd: cgroup directory :return: [] values or None when FAILED
-
initialize
(modules)[source]¶ Initializes object for use.
Parameters: modules – Array of all available cgroup modules.
-
is_cgroup
(pid, pwd)[source]¶ Checks if the ‘pid’ process is in ‘pwd’ cgroup :param pid: pid of the process :param pwd: cgroup directory :return: 0 when is ‘pwd’ member
-
is_root_cgroup
(pid)[source]¶ Checks if the ‘pid’ process is in root cgroup (WO cgroup) :param pid: pid of the process :return: 0 when is ‘root’ member
-
mk_cgroup
(pwd=None, cgroup=None)[source]¶ Creates new temporary cgroup :param pwd: where to create this cgroup (default: self.root) :param cgroup: desired cgroup name :return: last cgroup index
-
mk_cgroup_cgcreate
(pwd=None, cgroup=None)[source]¶ Make a cgroup by cgcreate command
Params: cgroup: Maked cgroup name Returns: last cgroup index
-
set_cgroup
(pid, pwd=None)[source]¶ Sets cgroup membership :param pid: pid of the process :param pwd: cgroup directory
-
set_property
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the property value :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value
-
set_property_h
(prop, value, pwd=None, check=True, checkprop=None)[source]¶ Sets the one-line property value concerning the K,M,G postfix :param prop: property name (file) :param value: desired value :param pwd: cgroup directory :param check: check the value after setup / override checking value :param checkprop: override prop when checking the value
-
-
class
autotest.client.shared.utils_cgroup.
CgroupModules
(mountdir=None)[source]¶ Bases:
object
Handles the list of different cgroup filesystems.
-
autotest.client.shared.utils_cgroup.
cgconfig_exists
()[source]¶ Check if cgconfig is available on the host or perhaps systemd is used
-
autotest.client.shared.utils_cgroup.
get_all_controllers
()[source]¶ Get all controllers used in system
Returns: all used controllers(controller_list)
-
autotest.client.shared.utils_cgroup.
get_cgroup_mountpoint
(controller)[source]¶ Get desired controller’s mountpoint
@controller: Desired controller :return: controller’s mountpoint
-
autotest.client.shared.utils_cgroup.
get_load_per_cpu
(_stats=None)[source]¶ Gather load per cpu from /proc/stat :param _stats: previous values :return: list of diff/absolute values of CPU times [SUM, CPU1, CPU2, …]
-
autotest.client.shared.utils_cgroup.
resolve_task_cgroup_path
(pid, controller)[source]¶ Resolving cgroup mount path of a particular task
Params: pid : process id of a task for which the cgroup path required Params: controller: takes one of the controller names in controller list Returns: resolved path for cgroup controllers of a given pid
-
autotest.client.shared.utils_cgroup.
service_cgconfig_control
(action)[source]¶ Cgconfig control by action.
If cmd executes successfully, return True, otherwise return False. If the action is status, return True when it’s running, otherwise return False. To check if the cgconfig stuff is available, use action “exists”.
@ param action: start|stop|status|restart|condrestart
utils_koji
Module¶
-
class
autotest.client.shared.utils_koji.
KojiClient
(cmd=None)[source]¶ Bases:
object
Stablishes a connection with the build system, either koji or brew.
This class provides convenience methods to retrieve information on packages and the packages themselves hosted on the build system. Packages should be specified in the KojiPgkSpec syntax.
-
CMD_LOOKUP_ORDER
= ['/usr/bin/brew', '/usr/bin/koji']¶
-
CONFIG_MAP
= {'/usr/bin/brew': '/etc/brewkoji.conf', '/usr/bin/koji': '/etc/koji.conf'}¶
-
get_default_command
()[source]¶ Looks up for koji or brew “binaries” on the system
Systems with plain koji usually don’t have a brew cmd, while systems with koji, have both koji and brew utilities. So we look for brew first, and if found, we consider that the system is configured for brew. If not, we consider this is a system with plain koji.
Returns: either koji or brew command line executable path, or None
-
get_pkg_info
(pkg)[source]¶ Returns information from Koji on the package
Parameters: pkg (KojiPkgSpec) – information about the package, as a KojiPkgSpec instance Returns: information from Koji about the specified package
-
get_pkg_rpm_file_names
(pkg, arch=None)[source]¶ Gets the file names for the RPM packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_rpm_info
(pkg, arch=None)[source]¶ Returns a list of information on the RPM packages found on koji
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_rpm_names
(pkg, arch=None)[source]¶ Gets the names for the RPM packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkg_urls
(pkg, arch=None)[source]¶ Gets the urls for the packages specified in pkg
Parameters: - pkg (KojiPkgSpec) – a package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_pkgs
(pkg, dst_dir, arch=None)[source]¶ Download the packages
Parameters: - pkg (KojiPkgSpec) – a package specification
- dst_dir (string) – the destination directory, where the downloaded packages will be saved on
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_scratch_pkg_urls
(pkg, arch=None)[source]¶ Gets the urls for the scratch packages specified in pkg
Parameters: - pkg (KojiScratchPkgSpec) – a scratch package specification
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_scratch_pkgs
(pkg, dst_dir, arch=None)[source]¶ Download the packages from a scratch build
Parameters: - pkg (KojiScratchPkgSpec) – a scratch package specification
- dst_dir (string) – the destination directory, where the downloaded packages will be saved on
- arch (string) – packages built for this architecture, but also including architecture independent (noarch) packages
-
get_session_options
()[source]¶ Filter only options necessary for setting up a cobbler client session
Returns: only the options used for session setup
-
is_config_valid
()[source]¶ Checks if the currently set koji configuration is valid
Returns: True or False
-
is_pkg_spec_build_valid
(pkg)[source]¶ Checks if build is valid on Koji
Parameters: pkg – a Pkg instance
-
is_pkg_spec_tag_valid
(pkg)[source]¶ Checks if tag is valid on Koji
Parameters: pkg (KojiPkgSpec) – a package specification
-
-
class
autotest.client.shared.utils_koji.
KojiDirIndexParser
[source]¶ Bases:
HTMLParser.HTMLParser
Parser for HTML directory index pages, specialized to look for RPM links
-
class
autotest.client.shared.utils_koji.
KojiPkgSpec
(text='', tag=None, build=None, package=None, subpackages=[])[source]¶ Bases:
object
A package specification syntax parser for Koji
This holds information on either tag or build, and packages to be fetched from koji and possibly installed (features external do this class).
New objects can be created either by providing information in the textual format or by using the actual parameters for tag, build, package and sub- packages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.
The following sets of examples are interchangeable. Specifying all packages part of build number 1000:
>>> from kvm_utils import KojiPkgSpec >>> pkg = KojiPkgSpec('1000')
>>> pkg = KojiPkgSpec(build=1000)
Specifying only a subset of packages of build number 1000:
>>> pkg = KojiPkgSpec('1000:kernel,kernel-devel')
>>> pkg = KojiPkgSpec(build=1000, subpackages=['kernel', 'kernel-devel'])
Specifying the latest build for the ‘kernel’ package tagged with ‘dist-f14’:
>>> pkg = KojiPkgSpec('dist-f14:kernel')
>>> pkg = KojiPkgSpec(tag='dist-f14', package='kernel')
Specifying the ‘kernel’ package using the default tag:
>>> kvm_utils.set_default_koji_tag('dist-f14') >>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')
Specifying the ‘kernel’ package using the default tag:
>>> kvm_utils.set_default_koji_tag('dist-f14') >>> pkg = KojiPkgSpec('kernel')
>>> pkg = KojiPkgSpec(package='kernel')
If you do not specify a default tag, and give a package name without an explicit tag, your package specification is considered invalid:
>>> print kvm_utils.get_default_koji_tag() None >>> print kvm_utils.KojiPkgSpec('kernel').is_valid() False
>>> print kvm_utils.KojiPkgSpec(package='kernel').is_valid() False
-
SEP
= ':'¶
-
describe
()[source]¶ Describe this package specification, in a human friendly way
Returns: package specification description
-
is_valid
()[source]¶ Checks if this package specification is valid.
Being valid means that it has enough and not conflicting information. It does not validate that the packages specified actually existe on the Koji server.
Returns: True or False
-
parse
(text)[source]¶ Parses a textual representation of a package specification
Parameters: text (string) – textual representation of a package in koji
-
to_text
()[source]¶ Return the textual representation of this package spec
The output should be consumable by parse() and produce the same package specification.
We find that it’s acceptable to put the currently set default tag as the package explicit tag in the textual definition for completeness.
Returns: package specification in a textual representation
-
-
class
autotest.client.shared.utils_koji.
KojiScratchPkgSpec
(text='', user=None, task=None, subpackages=[])[source]¶ Bases:
object
A package specification syntax parser for Koji scratch builds
This holds information on user, task and subpackages to be fetched from koji and possibly installed (features external do this class).
New objects can be created either by providing information in the textual format or by using the actual parameters for user, task and subpackages. The textual format is useful for command line interfaces and configuration files, while using parameters is better for using this in a programatic fashion.
This package definition has a special behaviour: if no subpackages are specified, all packages of the chosen architecture (plus noarch packages) will match.
The following sets of examples are interchangeable. Specifying all packages from a scratch build (whose task id is 1000) sent by user jdoe:
>>> from kvm_utils import KojiScratchPkgSpec >>> pkg = KojiScratchPkgSpec('jdoe:1000')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000)
Specifying some packages from a scratch build whose task id is 1000, sent by user jdoe:
>>> pkg = KojiScratchPkgSpec('jdoe:1000:kernel,kernel-devel')
>>> pkg = KojiScratchPkgSpec(user=jdoe, task=1000, subpackages=['kernel', 'kernel-devel'])
-
SEP
= ':'¶
-
-
class
autotest.client.shared.utils_koji.
RPMFileNameInfo
(filename)[source]¶ Simple parser for RPM based on information present on the filename itself
utils_memory
Module¶
-
autotest.client.shared.utils_memory.
drop_caches
()[source]¶ Writes back all dirty pages to disk and clears all the caches.
-
autotest.client.shared.utils_memory.
get_buddy_info
(chunk_sizes, nodes='all', zones='all')[source]¶ Get the fragement status of the host. It use the same method to get the page size in buddyinfo. 2^chunk_size * page_size The chunk_sizes can be string make up by all orders that you want to check splited with blank or a mathematical expression with ‘>’, ‘<’ or ‘=’. For example: The input of chunk_size could be: “0 2 4” And the return will be: {‘0’: 3, ‘2’: 286, ‘4’: 687} if you are using expression: “>=9” the return will be: {‘9’: 63, ‘10’: 225}
Parameters: - chunk_size (string) – The order number shows in buddyinfo. This is not the real page size.
- nodes (string) – The numa node that you want to check. Default value is all
- zones (string) – The memory zone that you want to check. Default value is all
Returns: A dict using the chunk_size as the keys
Return type:
-
autotest.client.shared.utils_memory.
read_from_numa_maps
(pid, key)[source]¶ Get the process numa related info from numa_maps. This function only use to get the numbers like anon=1.
Parameters: - pid (String) – Process id
- key (String) – The item you want to check from numa_maps
Returns: A dict using the address as the keys
Return type:
-
autotest.client.shared.utils_memory.
read_from_smaps
(pid, key)[source]¶ Get specific item value from the smaps of a process include all sections.
Parameters: - pid (String) – Process id
- key (String) – The item you want to check from smaps
Returns: The value of the item in kb
Return type:
version
Module¶
Based on work from Douglas Creager <dcreager@dcreager.net>
Gets the current version number. If possible, this is the output of “git describe”, modified to conform to the versioning scheme that setuptools uses. If “git describe” returns an error (most likely because we’re in an unpacked copy of a release tarball, rather than in a git working copy), then we fall back on reading the contents of the RELEASE-VERSION file.
To use this script, simply import it your setup.py file, and use the results of get_version() as your package version:
from autotest.client.shared import version
- setup(
- version=get_version(), . . .
)
This will automatically update the RELEASE-VERSION file, if necessary. Note that the RELEASE-VERSION file should not be checked into git; please add it to your top-level .gitignore file.
You’ll probably want to distribute the RELEASE-VERSION file in your sdist tarballs; to do this, just create a MANIFEST.in file that contains the following line:
include RELEASE-VERSION