Collection of Useful Python Scripts for System Operations

Python developers often need quick scripts to automate routine tasks. This article compiles several practical Python utilities.

Fix unzip filename encoding on Linux

This script extracts files from a zip archive, handling filenames encoded in GBK to avoid garbled characters.

<code><span><span>import</span> os</span></code><code><span><span>import</span> sys</span></code><code><span><span>import</span> zipfile</span></code><code><span><span>import</span> argparse</span></code><code><span><span>s = <span>'\x1b[%d;%dm%s\x1b[0m'</span></span></code><code><span><br/></span></code><code><span><span>def</span> <span>unzip</span><span>(path)</span>:</span></code><code><span><br/></span></code><code><span>    file = zipfile.ZipFile(path,<span>"r"</span>)</span></code><code><span>    <span>if</span> args.secret:</span></code><code><span>        file.setpassword(args.secret)</span></code><code><span><br/></span></code><code><span>    <span>for</span> name <span>in</span> file.namelist():</span></code><code><span>        <span>try</span>:</span></code><code><span>            utf8name=name.decode(<span>'gbk'</span>)</span></code><code><span>            pathname = os.path.dirname(utf8name)</span></code><code><span>        <span>except</span>:</span></code><code><span>            utf8name=name</span></code><code><span>            pathname = os.path.dirname(utf8name)</span></code><code><span><br/></span></code><code><span>        <span>#print s % (1, 92, '  &gt;&gt; extracting:'), utf8name</span></span></code><code><span>        <span>#pathname = os.path.dirname(utf8name)</span></span></code><code><span>        <span>if</span> <span>not</span> os.path.exists(pathname) <span>and</span> pathname != <span>""</span>:</span></code><code><span>            os.makedirs(pathname)</span></code><code><span>        data = file.read(name)</span></code><code><span>        <span>if</span> <span>not</span> os.path.exists(utf8name):</span></code><code><span>            <span>try</span>:</span></code><code><span>                fo = open(utf8name, <span>"w"</span>)</span></code><code><span>                fo.write(data)</span></code><code><span>                fo.close</span></code><code><span>            <span>except</span>:</span></code><code><span>                <span>pass</span></span></code><code><span>    file.close()</span></code><code><span><br/></span></code><code><span><span>def</span> <span>main</span><span>(argv)</span>:</span></code><code><span>    <span>######################################################</span></span></code><code><span>    <span># for argparse</span></span></code><code><span>    p = argparse.ArgumentParser(description=<span>'解决unzip乱码'</span>)</span></code><code><span>    p.add_argument(<span>'xxx'</span>, type=str, nargs=<span>'*'</span>, \</span></code><code><span>        help=<span>'命令对象.'</span>)</span></code><code><span>    p.add_argument(<span>'-s'</span>, <span>'--secret'</span>, action=<span>'store'</span>, \</span></code><code><span>        default=<span>None</span>, help=<span>'密码'</span>)</span></code><code><span>    <span>global</span> args</span></code><code><span>    args = p.parse_args(argv[<span>1</span>:])</span></code><code><span>    xxx = args.xxx</span></code><code><span><br/></span></code><code><span>    <span>for</span> path <span>in</span> xxx:</span></code><code><span>        <span>if</span> path.endswith(<span>'.zip'</span>):</span></code><code><span>            <span>if</span> os.path.exists(path):</span></code><code><span>                <span>print</span> s % (<span>1</span>, <span>97</span>, <span>'  ++ unzip:'</span>), path</span></code><code><span>                unzip(path)</span></code><code><span>            <span>else</span>:</span></code><code><span>                <span>print</span> s % (<span>1</span>, <span>91</span>, <span>'  !! file doesn\'t exist.'</span>), path</span></code><code><span>        <span>else</span>:</span></code><code><span>            <span>print</span> s % (<span>1</span>, <span>91</span>, <span>'  !! file isn\'t a zip file.'</span>), path</span></code><code><span><br/></span></code><code><span><span>if</span> __name__ == <span>'__main__'</span>:</span></code><code><span>    argv = sys.argv</span></code><code><span>    main(argv)</span></code>

Count lines of code in the current root directory

This script recursively walks through a directory tree and counts the total number of non‑empty lines in files with extensions specified in a whitelist (e.g., .cpp, .h).

<code><span><span># coding=utf-8</span></span></code><code><span><span>import</span> os</span></code><code><span><span>import</span> time</span></code><code><span><span># 设定根目录</span></span></code><code><span>basedir = <span>'./'</span></span></code><code><span>filelists = []</span></code><code><span><span># 指定想要统计的文件类型</span></span></code><code><span>whitelist = [<span>'cpp'</span>, <span>'h'</span>]</span></code><code><span><span>#遍历文件, 递归遍历文件夹中的所有</span></span></code><code><span><span><span>def</span> <span>getFile</span><span>(basedir)</span>:</span></span></code><code><span>    <span>global</span> filelists</span></code><code><span>    <span>for</span> parent,dirnames,filenames <span>in</span> os.walk(basedir):</span></code><code><span>        <span>for</span> filename <span>in</span> filenames:</span></code><code><span>            ext = filename.split(<span>'.'</span>)[<span>-1</span>]</span></code><code><span>            <span>#只统计指定的文件类型，略过一些log和cache文件</span></span></code><code><span>            <span>if</span> ext <span>in</span> whitelist:</span></code><code><span>                filelists.append(os.path.join(parent,filename))</span></code><code><span><span>#统计一个的行数</span></span></code><code><span><span><span>def</span> <span>countLine</span><span>(fname)</span>:</span></span></code><code><span>    count = <span>0</span></span></code><code><span>    <span># 把文件做二进制看待,read.</span></span></code><code><span>    <span>for</span> file_line <span>in</span> open(fname, <span>'rb'</span>).readlines():</span></code><code><span>        <span>if</span> file_line != <span>''</span> <span>and</span> file_line != <span>'\n'</span>: <span>#过滤掉空行</span></span></code><code><span>            count += <span>1</span></span></code><code><span>    <span>print</span> (fname + <span>'----'</span> , count)</span></code><code><span>    <span>return</span> count</span></code><code><span><span>if</span> __name__ == <span>'__main__'</span> :</span></code><code><span>    startTime = time.clock()</span></code><code><span>    getFile(basedir)</span></code><code><span>    totalline = <span>0</span></span></code><code><span>    <span>for</span> filelist <span>in</span> filelists:</span></code><code><span>        totalline = totalline + countLine(filelist)</span></code><code><span>    <span>print</span> (<span>'total lines:'</span>,totalline)</span></code><code><span>    <span>print</span> (<span>'Done! Cost Time: %0.2f second'</span> % (time.clock() - startTime))</span></code>

Scan current directory and sub‑directories and display size

This utility walks a directory tree, sums the sizes of all files, and prints the total size in various units.

<code><span>import os</span></code><code><span>import sys      </span></code><code><span>try:</span></code><code><span>    directory = sys.argv[1]   </span></code><code><span>except IndexError:</span></code><code><span>    sys.exit(<span>"Must provide an argument."</span>)</span></code><code><span><br/></span></code><code><span>dir_size = 0   </span></code><code><span>fsizedicr = {<span>'Bytes'</span>: 1,</span></code><code><span>             <span>'Kilobytes'</span>: <span>float</span>(1) / 1024,</span></code><code><span>             <span>'Megabytes'</span>: <span>float</span>(1) / (1024 * 1024),</span></code><code><span>             <span>'Gigabytes'</span>: <span>float</span>(1) / (1024 * 1024 * 1024)}</span></code><code><span><span>for</span> (path, <span>dirs</span>, files) <span>in</span> os.walk(directory):      </span></code><code><span>    <span>for</span> file <span>in</span> files:                              </span></code><code><span>        filename = os.path.join(path, file)</span></code><code><span>        dir_size += os.path.getsize(filename)       </span></code><code><span><br/></span></code><code><span>fsizeList = [str(round(fsizedicr[key] * dir_size, 2)) + <span>" "</span> + key <span>for</span> key <span>in</span> fsizedicr] </span></code><code><span><br/></span></code><code><span><span>if</span> dir_size == 0: <span>print</span> (<span>"File Empty"</span>) </span></code><code><span><span>else</span>:</span></code><code><span>  <span>for</span> units <span>in</span> sorted(fsizeList)[::-1]: </span></code><code><span>      <span>print</span> (<span>"Folder Size: "</span> + units)</span></code>

Move files older than 240 days from source to destination

This script moves files whose modification time exceeds a specified number of days (default 240) from a source directory to a target directory.

<code><span><span>import</span> shutil</span></code><code><span><span>import</span> sys</span></code><code><span><span>import</span> time</span></code><code><span><span>import</span> os</span></code><code><span><span>import</span> argparse</span></code><code><span><br/></span></code><code><span>usage = <span>'python move_files_over_x_days.py -src [SRC] -dst [DST] -days [DAYS]'</span></span></code><code><span>description = <span>'Move files from src to dst if they are older than a certain number of days.  Default is 240 days'</span></span></code><code><span><br/></span></code><code><span>args_parser = argparse.ArgumentParser(usage=usage, description=description)</span></code><code><span>args_parser.add_argument(<span>'-src'</span>, <span>'--src'</span>, type=str, nargs=<span>'?'</span>, default=<span>'.'</span>, help=<span>'(OPTIONAL) Directory where files will be moved from. Defaults to current directory'</span>)</span></code><code><span>args_parser.add_argument(<span>'-dst'</span>, <span>'--dst'</span>, type=str, nargs=<span>'?'</span>, required=<span>True</span>, help=<span>'(REQUIRED) Directory where files will be moved to.'</span>)</span></code><code><span>args_parser.add_argument(<span>'-days'</span>, <span>'--days'</span>, type=int, nargs=<span>'?'</span>, default=<span>240</span>, help=<span>'(OPTIONAL) Days value specifies the minimum age of files to be moved. Default is 240.'</span>)</span></code><code><span>args = args_parser.parse_args()</span></code><code><span><br/></span></code><code><span><span>if</span> args.days &lt; <span>0</span>:</span></code><code><span>  args.days = <span>0</span></span></code><code><span><br/></span></code><code><span>src = args.src  <span># 设置源目录</span></span></code><code><span>dst = args.dst  <span># 设置目标目录</span></span></code><code><span>days = args.days <span># 设置天数</span></span></code><code><span>now = time.time()  <span># 获得当前时间</span></span></code><code><span><br/></span></code><code><span><span>if</span> <span>not</span> os.path.exists(dst):</span></code><code><span>  os.mkdir(dst)</span></code><code><span><br/></span></code><code><span><span>for</span> f <span>in</span> os.listdir(src):  <span># 遍历源目录所有文件</span></span></code><code><span>    <span>if</span> os.stat(f).st_mtime &lt; now - days * <span>86400</span>:  <span># 判断是否超过240天</span></span></code><code><span>        <span>if</span> os.path.isfile(f):  <span># 检查是否是文件</span></span></code><code><span>            shutil.move(f, dst)  <span># 移动文件</span></span></code>

Scan script directory and count different script types

This snippet zips log files and moves the archives, demonstrating basic file manipulation.

<code><span><span>import</span> os                                  </span></code><code><span><span>import</span> shutil                                </span></code><code><span>from time <span>import</span> strftime                        </span></code><code><span><br/></span></code><code><span>logsdir=<span>"c:\logs\puttylogs"</span>                      </span></code><code><span>zipdir=<span>"c:\logs\puttylogs\zipped_logs"</span>              </span></code><code><span>zip_program=<span>"zip.exe"</span>                        </span></code><code><span><br/></span></code><code><span><span>for</span> files <span>in</span> os.listdir(logsdir):                    </span></code><code><span>  <span>if</span> files.endswith(<span>".log"</span>):                    </span></code><code><span>    files1=files+<span>"."</span>+strftime(<span>"%Y-%m-%d"</span>)+<span>".zip"</span>    </span></code><code><span>    os.chdir(logsdir)                         </span></code><code><span>    os.system(zip_program + <span>" "</span> +  files1 +<span>" "</span>+ files)  </span></code><code><span>    shutil.move(files1, zipdir)                   </span></code><code><span>    os.remove(files)                          </span></code>

Download LeetCode algorithm problems

This script fetches problem information from LeetCode, saves it as a pickle file, and can export the data to a formatted text file.

<code><span><span>import</span> sys</span></code><code><span><span>import</span> re</span></code><code><span><span>import</span> os</span></code><code><span><span>import</span> argparse</span></code><code><span><span>import</span> requests</span></code><code><span><span>from</span> lxml <span>import</span> html <span>as</span> lxml_html</span></code><code><span><br/></span></code><code><span><span>try</span>:</span></code><code><span>    <span>import</span> html</span></code><code><span><span>except</span> ImportError:</span></code><code><span>    <span>import</span> HTMLParser</span></code><code><span>    html = HTMLParser.HTMLParser()</span></code><code><span><br/></span></code><code><span><span>try</span>:</span></code><code><span>    <span>import</span> cPickle <span>as</span> pk</span></code><code><span><span>except</span> ImportError:</span></code><code><span>    <span>import</span> pickle <span>as</span> pk</span></code><code><span><br/></span></code><code><span><span><span>class</span> <span>LeetcodeProblems</span><span>(object)</span>:</span></span></code><code><span>    <span><span>def</span> <span>get_problems_info</span><span>(self)</span>:</span></span></code><code><span>        leetcode_url = <span>'https://leetcode.com/problemset/algorithms'</span></span></code><code><span>        res = requests.get(leetcode_url)</span></code><code><span>        <span>if</span> <span>not</span> res.ok:</span></code><code><span>            print(<span>'request error'</span>)</span></code><code><span>            sys.exit()</span></code><code><span>        cm = res.text</span></code><code><span>        cmt = cm.split(<span>'tbody&gt;'</span>)[<span>-2</span>]</span></code><code><span>        indexs = re.findall(<span>r'&lt;td&gt;(\d+)&lt;/td&gt;'</span>, cmt)</span></code><code><span>        problem_urls = [<span>'https://leetcode.com'</span> + url \
<span>                        <span>for</span> url <span>in</span> re.findall(</span></code><code><span>                            <span>r'&lt;a href="(/problems/.+?)"'</span>, cmt)]</span></code><code><span>        levels = re.findall(<span>r"&lt;td value='\d*'&gt;(.+?)&lt;/td&gt;"</span>, cmt)</span></code><code><span>        tinfos = zip(indexs, levels, problem_urls)</span></code><code><span>        <span>assert</span> (len(indexs) == len(problem_urls) == len(levels))</span></code><code><span>        infos = []</span></code><code><span>        <span>for</span> info <span>in</span> tinfos:</span></code><code><span>            res = requests.get(info[<span>-1</span>])</span></code><code><span>            <span>if</span> <span>not</span> res.ok:</span></code><code><span>                print(<span>'request error'</span>)</span></code><code><span>                sys.exit()</span></code><code><span>            tree = lxml_html.fromstring(res.text)</span></code><code><span>            title = tree.xpath(<span>'//meta[@property="og:title"]/@content'</span>)[<span>0</span>]</span></code><code><span>            description = tree.xpath(<span>'//meta[@property="description"]/@content'</span>)</span></code><code><span>            <span>if</span> <span>not</span> description:</span></code><code><span>                description = tree.xpath(<span>'//meta[@property="og:description"]/@content'</span>)[<span>0</span>]</span></code><code><span>            <span>else</span>:</span></code><code><span>                description = description[<span>0</span>]</span></code><code><span>            description = html.unescape(description.strip())</span></code><code><span>            tags = tree.xpath(<span>'//div[@id="tags"]/following::a[@class="btn btn-xs btn-primary"]/text()'</span>)</span></code><code><span>            infos.append(</span></code><code><span>                {</span></code><code><span>                    <span>'title'</span>: title,</span></code><code><span>                    <span>'level'</span>: info[<span>1</span>],</span></code><code><span>                    <span>'index'</span>: int(info[<span>0</span>]),</span></code><code><span>                    <span>'description'</span>: description,</span></code><code><span>                    <span>'tags'</span>: tags</span></code><code><span>                }</span></code><code><span>            )</span></code><code><span>        <span>with</span> open(<span>'leecode_problems.pk'</span>, <span>'wb'</span>) <span>as</span> g:</span></code><code><span>            pk.dump(infos, g)</span></code><code><span>        <span>return</span> infos</span></code><code><span><br/></span></code><code><span>    <span><span>def</span> <span>to_text</span><span>(self, pm_infos)</span>:</span></span></code><code><span>        <span>if</span> self.args.index:</span></code><code><span>            key = <span>'index'</span></span></code><code><span>        <span>elif</span> self.args.title:</span></code><code><span>            key = <span>'title'</span></span></code><code><span>        <span>elif</span> self.args.tag:</span></code><code><span>            key = <span>'tags'</span></span></code><code><span>        <span>elif</span> self.args.level:</span></code><code><span>            key = <span>'level'</span></span></code><code><span>        <span>else</span>:</span></code><code><span>            key = <span>'index'</span></span></code><span>... (truncated for brevity) ...</span>

Convert Markdown to HTML

This class reads a Markdown file, optionally injects a CSS stylesheet, converts the content to HTML using the markdown library, and writes a prettified HTML file.

<code><span><span>import</span> sys</span></code><code><span><span>import</span> os</span></code><code><span><br/></span></code><code><span><span>from</span> bs4 <span>import</span> BeautifulSoup</span></code><code><span><span>import</span> markdown</span></code><code><span><br/></span></code><code><span><span>class</span> <span>MarkdownToHtml</span>:</span></span></code><code><span>    headTag = <span>'&lt;head&gt;&lt;meta charset="utf-8" /&gt;&lt;/head&gt;'</span></span></code><code><span>    <span>def</span> __init__(self,cssFilePath = None):</span></code><code><span>        <span>if</span> cssFilePath != <span>None</span>:</span></code><code><span>            self.genStyle(cssFilePath)</span></code><code><span>    <span>def</span> genStyle(self,cssFilePath):</span></code><code><span>        <span>with</span> open(cssFilePath,<span>'r'</span>) <span>as</span> f:</span></code><code><span>            cssString = f.read()</span></code><code><span>        self.headTag = self.headTag[:<span>-7</span>] + <span>'&lt;style type="text/css"&gt;{}&lt;/style&gt;'</span>.format(cssString) + self.headTag[<span>-7</span>:]</span></code><code><span>    <span>def</span> markdownToHtml(self, sourceFilePath, destinationDirectory = None, outputFileName = None):</span></code><code><span>        <span>if</span> <span>not</span> destinationDirectory:</span></code><code><span>            destinationDirectory = os.path.dirname(os.path.abspath(sourceFilePath))</span></code><code><span>        <span>if</span> <span>not</span> outputFileName:</span></code><code><span>            outputFileName = os.path.splitext(os.path.basename(sourceFilePath))[<span>0</span>] + <span>'.html'</span></span></code><code><span>        <span>if</span> destinationDirectory[<span>-1</span>] != <span>'/'</span>:</span></code><code><span>            destinationDirectory += <span>'/'</span></span></code><code><span>        <span>with</span> open(sourceFilePath,<span>'r'</span>, encoding=<span>'utf8'</span>) <span>as</span> f:</span></code><code><span>            markdownText = f.read()</span></code><code><span>        rawHtml = self.headTag + markdown.markdown(markdownText,output_format=<span>'html5'</span>)</span></code><code><span>        beautifyHtml = BeautifulSoup(rawHtml,<span>'html5lib'</span>).prettify()</span></code><code><span>        <span>with</span> open(destinationDirectory + outputFileName, <span>'w'</span>, encoding=<span>'utf8'</span>) <span>as</span> f:</span></code><code><span>            f.write(beautifyHtml)</span></code><code><span>... (truncated for brevity) ...</span></code>

Detect and convert text file encodings

This utility uses chardet to detect the encoding of one or more text files and optionally converts them to a target encoding.

<code><span><span>import</span> sys</span></code><code><span><span>import</span> os</span></code><code><span><span>import</span> argparse</span></code><code><span><span>from</span> chardet.universaldetector <span>import</span> UniversalDetector</span></code><code><span><br/></span></code><code><span>parser = argparse.ArgumentParser(description = <span>'文本文件编码检测与转换'</span>)</span></code><code><span>parser.add_argument(<span>'filePaths'</span>, nargs = <span>'+'</span>,</span></code><code><span>                   help = <span>'检测或转换的文件路径'</span>)</span></code><code><span>parser.add_argument(<span>'-e'</span>, <span>'--encoding'</span>, nargs = <span>'?'</span>, const = <span>'UTF-8'</span>,</span></code><code><span>                   help = <span>'''目标编码。支持的编码有：ASCII, (Default) UTF-8 ...'''</span>)</span></code><code><span>parser.add_argument(<span>'-o'</span>, <span>'--output'</span>,</span></code><code><span>                   help = <span>'输出目录'</span>)</span></code><code><span># 解析参数，得到一个 Namespace 对象</span></code><code><span>args = parser.parse_args()</span></code><code><span># 输出目录不为空即视为开启转换, 若未指定转换编码，则默认为 UTF-8</span></code><code><span>if args.output != <span>None</span>:</span></code><code><span>    <span>if</span> <span>not</span> args.encoding:</span></code><code><span>        args.encoding = <span>'UTF-8'</span></span></code><code><span>    <span>if</span> <span>not</span> os.path.isdir(args.output):</span></code><code><span>        print(<span>'Invalid Directory: '</span> + args.output)</span></code><code><span>        sys.exit()</span></code><code><span>    <span>else</span>:</span></code><code><span>        <span>if</span> args.output[<span>-1</span>] != <span>'/'</span>:</span></code><code><span>            args.output += <span>'/'</span></span></code><code><span>detector = UniversalDetector()</span></code><code><span>print()</span></code><code><span>print(<span>'Encoding (Confidence)'</span>,<span>':'</span>,<span>'File path'</span>)</span></code><code><span>for filePath in args.filePaths:</span></code><code><span>    <span>if</span> <span>not</span> os.path.isfile(filePath):</span></code><code><span>        print(<span>'Invalid Path: '</span> + filePath)</span></code><code><span>        <span>continue</span></span></code><code><span>    detector.reset()</span></code><code><span>    for each in open(filePath, <span>'rb'</span>):</span></code><code><span>        detector.feed(each)</span></code><code><span>        <span>if</span> detector.done:</span></code><code><span>            <span>break</span></span></code><code><span>    detector.close()</span></code><code><span>    charEncoding = detector.result[<span>'encoding'</span>]</span></code><code><span>    confidence = detector.result[<span>'confidence'</span>]</span></code><code><span>    <span>if</span> charEncoding <span>is</span> <span>None</span>:</span></code><code><span>        charEncoding = <span>'Unknown'</span></span></code><code><span>        confidence = <span>0.99</span></span></code><code><span>    print(<span>'{} {:&gt;12} : {}'</span>.format(charEncoding.rjust(<span>8</span>),</span></code><code><span>        <span>'('</span>+str(confidence*<span>100</span>)+<span>'%)'</span>, filePath))</span></code><code><span>    <span>if</span> args.encoding <span>and</span> charEncoding != <span>'Unknown'</span> <span>and</span> confidence &gt; <span>0.6</span>:</span></code><code><span>        outputPath = args.output + os.path.basename(filePath) <span>if</span> args.output <span>else</span> filePath</span></code><code><span>        <span>with</span> open(filePath, <span>'r'</span>, encoding = charEncoding, errors = <span>'replace'</span>) <span>as</span> f:</span></code><code><span>            temp = f.read()</span></code><code><span>        <span>with</span> open(outputPath, <span>'w'</span>, encoding = args.encoding, errors = <span>'replace'</span>) <span>as</span> f:</span></code><code><span>            f.write(temp)</span></code>

The article also includes a promotional banner encouraging readers to follow the public account for free Python learning resources.