OS module
Introduction
1. ConfigParser() python module:
ConfigParser() python module: This module provides the ConfigParser class which implements a basic configuration language which provides a structure similar to what’s found in Microsoft Windows INI files. You can use this to write Python programs which can be customized by end users easily. For example. We need to import the configparser module. This module is used to handle configuration files in python:
config = configparser.ConfigParser()2. 'os' module
The os module in Python provides a wide range of functionalities for interacting with the operating system. Some of the key functionalities that can be useful in similar tasks to listing files in directories include:
| Function | Description | Example Code |
|---|---|---|
| os.getcwd() | Returns the current working directory. |
import os
current_directory = os.getcwd()
print(current_directory) |
| os.chdir(path) | Changes the current working directory to the specified path. |
import os
os.chdir('/path/to/directory') |
| os.listdir(path) | Returns a list of entries in the specified directory. |
import os
entries = os.listdir('/path/to/directory')
print(entries) |
| os.mkdir(path) | Creates a new directory at the specified path. |
import os
os.mkdir('/path/to/new_directory') |
| os.makedirs(path) | Creates a new directory and any necessary parent directories. |
import os
os.makedirs('/path/to/new_directory/sub_directory')
|
| os.remove(path) | Deletes the specified file. |
import os
os.remove('/path/to/file.txt') |
| os.rmdir(path) | Deletes the specified directory (only if it is empty). |
import os
os.rmdir('/path/to/empty_directory') |
| os.removedirs(path) | Deletes the specified directory and any empty parent directories. |
import os
os.removedirs('/path/to/empty_directory/sub_directory') |
| os.rename(src, dst) | Renames the file or directory from src to dst. |
import os
os.rename('/path/to/old_name.txt', '/path/to/new_name.txt') |
| os.replace(src, dst) | Renames the file or directory from src to dst. If dst exists, it will be replaced. |
import os
os.replace('/path/to/old_name.txt', '/path/to/new_name.txt') |
| os.path.basename(path) | Returns the base name of the pathname. |
import os
base_name = os.path.basename('/path/to/file.txt')
print(base_name) # Output: file.txt |
| os.path.dirname(path) | Returns the directory name of the pathname. |
import os
dir_name = os.path.dirname('/path/to/file.txt')
print(dir_name) # Output: /path/to |
| os.path.join(path, *paths) | Joins one or more path components intelligently. |
import os
full_path = os.path.join('/path/to', 'file.txt')
print(full_path) # Output: /path/to/file.txt |
| os.path.exists(path) | Returns True if the specified path exists, False otherwise. |
import os
exists = os.path.exists('/path/to/file.txt')
print(exists) |
| os.path.isabs(path) | Returns True if the specified path is an absolute path, False otherwise. |
import os
is_abs = os.path.isabs('/path/to/file.txt')
print(is_abs) |
| os.path.isfile(path) | Returns True if the specified path is a file, False otherwise. |
import os
is_file = os.path.isfile('/path/to/file.txt')
print(is_file) |
| os.path.isdir(path) | Returns True if the specified path is a directory, False otherwise. |
import os
is_dir = os.path.isdir('/path/to/directory')
print(is_dir) |
| os.path.getsize(path) | Returns the size of the specified file in bytes. |
import os
size = os.path.getsize('/path/to/file.txt')
print(size) |
| os.path.abspath(path) | Returns the absolute version of the specified path. |
import os
abs_path = os.path.abspath('file.txt')
print(abs_path) |
| os.environ | A dictionary representing the string environment. It can be used to access and modify environment variables. |
import os
path = os.environ.get('PATH')
print(path) |
| os.getenv(key, default=None) | Returns the value of the environment variable key if it exists, otherwise returns default. |
import os
home = os.getenv('HOME', '/default/path')
print(home) |
| os.putenv(key, value) | Sets the environment variable key to value. (Note: This function is deprecated in favor of directly modifying os.environ). |
import os
os.putenv('MY_VARIABLE', 'value') |
| os.unsetenv(key) | Deletes the environment variable key. (Note: This function is deprecated in favor of directly modifying os.environ). |
import os
os.unsetenv('MY_VARIABLE') |
| os.system(command) | Executes the command (a string) in a subshell. |
import os
os.system('echo Hello, World!') |
| os.popen(command) | Opens a pipe to or from the command. The return value is an open file object connected to the pipe. |
import os
with os.popen('ls -l') as stream:
output = stream.read()
print(output) |
| os.getpid() | Returns the current process ID. | import os
pid = os.getpid()
print(pid) |
| os.getppid() | Returns the parent process ID. | import os
ppid = os.getppid()
print(ppid) |
| os.fork() | Forks a child process. (Unix only) | import os
pid = os.fork()
if pid == 0:
print('Child process')
else:
print('Parent process') |
| os.execv(path, args) | Executes the executable file specified by path with the argument list args. | import os
os.execv('/bin/ls', ['ls', '-l']) |
| os.open(path, flags, mode=0o777) | Opens the file specified by path and returns a file descriptor. | import os
fd = os.open('/path/to/file.txt', os.O_RDWR | os.O_CREAT) |
| os.read(fd, n) | Reads at most n bytes from file descriptor fd. | import os
data = os.read(fd, 100)
print(data) |
| os.write(fd, str) | Writes the string str to file descriptor fd. | import os
os.write(fd, b'Hello, World!') |
| os.close(fd) | Closes the file descriptor fd. | import os
os.close(fd) |
| os.dup(fd) | Duplicates the file descriptor fd. | import os
new_fd = os.dup(fd) |
| os.dup2(fd, fd2) | Duplicates file descriptor fd to fd2. | import os
os.dup2(fd, fd2) |
| os.urandom(n) | Returns a string of n random bytes suitable for cryptographic use. | import os
random_bytes = os.urandom(16)
print(random_bytes) |
| os.path.split(path) | Splits the pathname into a pair (head, tail). | import os
head, tail = os.path.split('/path/to/file.txt')
print(head) # Output: /path/to
print(tail) # Output: file.txt |
Introduction to Configparser
Theconfigparser module in Python allows you to manage configuration files in a format similar to Windows INI files. These files contain sections, each with a number of key-value pairs, making them an ideal way to store configuration settings.
Basic Usage
Creating and Writing a Configuration File:
import configparser
# Create a ConfigParser object
config = configparser.ConfigParser()
# Add sections and settings
config['DEFAULT'] = {'ServerAliveInterval': '45',
'Compression': 'yes',
'CompressionLevel': '9'}
config['bitbucket.org'] = {'User': 'hg'}
config['topsecret.server.com'] = {'Host Port': '50022', 'ForwardX11': 'no'}
# Write to a configuration file
with open('config.ini', 'w') as configfile:
config.write(configfile)
config.ini file:
# Create the config.ini content based on the given Python code
config_content = """
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
[bitbucket.org]
User = hg
[topsecret.server.com]
Host Port = 50022
ForwardX11 = no
"""
Reading a Configuration File
import configparser
# Create a ConfigParser object
config = configparser.ConfigParser()
# Read the configuration file
config.read('config.ini')
# Access data
server_alive_interval = config['DEFAULT']['ServerAliveInterval']
compression = config['DEFAULT']['Compression']
user = config['bitbucket.org']['User']
print(server_alive_interval, compression, user)
Functions and Methods
Here are some key functions and methods provided by theconfigparser module:
| Method | Description | Example |
|---|---|---|
| read(filenames, encoding=None) | Reads and parses a list of filenames. Returns a list of successfully read files. | import configparser
config = configparser.ConfigParser()
config.read(['example.ini', 'other_config.ini']) |
| read_string(string) | Reads configuration from a given string. | import configparser
config = configparser.ConfigParser()
config.read_string("""
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
""") |
| read_dict(dictionary) | Reads configuration from a dictionary. | import configparser
config = configparser.ConfigParser()
config.read_dict({
'DEFAULT': {'ServerAliveInterval': '45', 'Compression': 'yes', 'CompressionLevel': '9'},
'bitbucket.org': {'User': 'hg'},
'topsecret.server.com': {'Host Port': '50022', 'ForwardX11': 'no'}
}) |
| sections() | Returns a list of section names, excluding [DEFAULT]. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
sections = config.sections()
print(sections) # ['bitbucket.org', 'topsecret.server.com'] |
| options(section) | Returns a list of option names for the given section. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
options = config.options('bitbucket.org')
print(options) # ['user'] |
| get(section, option, *, raw=False, vars=None, fallback=None) | Gets an option value for the given section. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
user = config.get('bitbucket.org', 'User')
print(user) # hg |
| getint(section, option, *, raw=False, vars=None, fallback=None) | Gets an integer option value. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
port = config.getint('topsecret.server.com', 'Host Port')
print(port) # 50022 |
| getfloat(section, option, *, raw=False, vars=None, fallback=None) | Gets a float option value. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
compression_level = config.getfloat('DEFAULT', 'CompressionLevel')
print(compression_level) # 9.0 |
| getboolean(section, option, *, raw=False, vars=None, fallback=None) | Gets a boolean option value. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
compression = config.getboolean('DEFAULT', 'Compression')
print(compression) # True |
| has_section(section) | Checks if the configuration has the specified section. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
has_bitbucket = config.has_section('bitbucket.org')
print(has_bitbucket) # True |
| add_section(section) | Adds a new section. | import configparser
config = configparser.ConfigParser()
config.add_section('new_section') |
| set(section, option, value) | Sets the value of an option. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
config.set('bitbucket.org', 'User', 'new_user') |
| remove_option(section, option) | Removes an option from the specified section. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
config.remove_option('bitbucket.org', 'User') |
| remove_section(section) | Removes a section. | import configparser
config = configparser.ConfigParser()
config.read('example.ini')
config.remove_section('bitbucket.org') |
| write(fileobject, space_around_delimiters=True) | Writes the configuration to a file. | import configparser
config = configparser.ConfigParser()
config['DEFAULT'] = {'ServerAliveInterval': '45', 'Compression': 'yes', 'CompressionLevel': '9'}
with open('example.ini', 'w') as configfile:
config.write(configfile) |
Applications
- Application Configuration: Store settings for applications, such as database connection strings, API keys, and other parameters that might change over time.
- User Preferences: Save user preferences for applications, like window size, theme, and other customizable settings.
- Environment Configuration: Manage different configurations for different environments (e.g., development, testing, production) without changing the application code.
- Logging Configuration: Store and manage logging configurations, including log levels, formats, and handlers.
- Test Configuration: Use configuration files to set parameters for testing frameworks, allowing for easy adjustments without code changes.
Example : Application Configuration
Consider an application that connects to a database.
import configparser
# Create a ConfigParser object
config = configparser.ConfigParser()
# Read the configuration file
config.read('app_config.ini')
# Access database settings
db_host = config.get('database', 'host')
db_port = config.getint('database', 'port')
db_user = config.get('database', 'user')
db_password = config.get('database', 'password')
print(f"Connecting to database at {db_host}:{db_port} with user {db_user}")
The app_config.ini: file is:
[database]
host = localhost
port = 5432
user = admin
password = secret
References
Some other interesting things to know:
- Visit my website on For Data, Big Data, Data-modeling, Datawarehouse, SQL, cloud-compute.
- Visit my website on Data engineering