optools/ref/python.tips_tricks_and_dirty_hacks

This is a collection of snippets that I tend to use frequently, or would need
in the future.

###############################################################################

To programmatically import a list of modules by name:
_____
import importlib

mods = ['os', 'sys', 'shutil', 'platform']

for m in mods:
    globals()[m] = importlib.import_module(m)
_____
you can then use them as if you did:

import os
import sys
import shutil
import platform
etc.

this breaks pep-8, but sometimes you need to programmatically import modules.

ref: https://www.python.org/dev/peps/pep-0008/#imports

###############################################################################

To programmatically install modules via pip if they aren't installed (BROKEN IN RECENT PIP VERSIONS; pip._internal.main() i think):

____
import importlib
import pip

# I don't  *think* pip/pypi is case-sensitive, but frequently module names are
# not the same as their package names. ugh.
# The key is the package name, the value is the module name. We use the above
# trick here to try to import and install if it fails.
mods = {'PyMySQL': 'pymysql',
        'Jinja2': 'jinja2',
        'psutil': None,  # We show off a little bit here with this, see below.
        'paramiko': None}  # ""

for m in mods.keys():
    modname = mods[m]
    if not modname:
        modname = m
    try:
        globals()[modname] = importlib.import_module(modname)
    except ImportError:
        # We use --user to avoid conflicts with the host's python system.
        # pip.main() accepts all of pip (commandline)'s args!
        pip.main(['install', '--user', m])
        try:
            globals()[modname] = importlib.import_module(modname)
        except ImportError:
            raise RuntimeError('Unable to install {0}!'.format(m))
____

###############################################################################

To convert an argparse set of parsed arguments into a dict from a class, you
simply do:
____
def GenArgs():
    args = argparse.ArgumentParser()
    # args here
    return(args)

def somefunc():
    args = vars(GenArgs().parse_args())
____
"args" in somefunc is a dict now.

###############################################################################

To dynamically allocate class parameters into constants from a dict (such as
from argparse - see above), do something like this:
____
class ClassName(object):
    def __init__(self, **kwargs):
        for i in kwargs.keys():
            setattr(self, i, kwargs[i])
----

###############################################################################

To store stdout and stderr to different files in a subprocess call:
----
with open('/tmp/test.o', 'w') as out, open('/tmp/test.e', 'w') as err:
    subprocess.run(['command'], stdout = out, stderr = err)
----
###############################################################################

To use optools logging lib (or other "shared" modules):
----
import os
import re
import importlib
spec = importlib.util.spec_from_file_location(
                                       'logger',
                                       '/opt/dev/optools/lib/python/logger.py')
logger = importlib.util.module_from_spec(spec)
spec.loader.exec_module(logger)
log = logger.log(name = 'project.name')
----

###############################################################################

# TODO #
https://stackoverflow.com/questions/10265193/python-can-a-class-act-like-a-module


###############################################################################

To issue an equivalent of "reset" command in linux, assuming console is ANSI-compat,

print('\x1bc', end = '')

###############################################################################


To get the default route via pyroute2,
----
import socket
from pyroute2 import IPDB
ip = IPDB()
def_rt = ip.routes['default']  # a route object
iface = ip.interfaces[def_rt.oif]  # an interface object. name is e.g. iface.ifname, IPs are in tuple-of-tuples iface.ipaddr, etc.
gw = def_rt.gateway  # etc.
ip.release()
----

###############################################################################


dropping privileges to non-root user (and restoring):
https://stackoverflow.com/questions/2699907/dropping-root-permissions-in-python
https://stackoverflow.com/questions/15705439/drop-root-privileges-for-certain-operations-in-python

NOTE: if you want to *remove the ability* to restore back to root privs, use os.setgid(running_gid) and os.setuid(running_uid) instead.

reference:
http://timetobleed.com/5-things-you-dont-know-about-user-ids-that-will-destroy-you/
https://nanopdf.com/download/setuid-demystified_pdf

----
import os, pwd, grp
import subprocess

my_uid = pwd.getpwuid(os.geteuid()).pw_uid
my_gid = grp.getgrgid(os.getegid()).gr_gid
my_grps = os.getgroups()
try:
    os.remove('/tmp/usertest')
    os.remove('/tmp/parenttest')
except:
    pass
if os.getuid() != 0:
    exit('need root')
running_uid = pwd.getpwnam(uid_name).pw_uid
running_gid = grp.getgrnam(gid_name).gr_gid
os.setgroups([])
os.setegid(running_gid)
os.seteuid(running_uid)
old_umask = os.umask(0o022)
subprocess.run(['touch', '/tmp/usertest'])
os.seteuid(my_uid)
os.setegid(my_gid)
os.setgroups(my_grps)
os.umask(old_umask)
subprocess.run(['touch', '/tmp/parenttest'])

###############################################################################


Getting remote sshd version

stdlib:
----
import socket

try:
    sock = socket.socket()
    sock.settimeout(10)
    sock.connect(('cylon', 22))
    version = sock.recv(64).decode('utf-8').strip()
    # version is a string like "SSH-2.0-OpenSSH_7.9"
    sock.close()
except Exception as e:
    # Obviously you can specify multiple exceptions to catch.
    raise RuntimeError(e)
----

paramiko (does not require auth):
----
import paramiko

t = paramiko.Transport(('cylon', 22))
t.connect()
t.remote_version
----