Python Utility for Reading Log Files and Generating Violin Charts with Plotly for Server Performance Testing

The author needed a way to process server performance test logs and visualize the data, so they created a Python utility that reads log files, filters out outliers, and stores the cleaned data for charting.

Below is the Python class for reading files, which defines a Fission class with methods to initialize, load data from a specified log file, filter values greater than 2 seconds, and build parallel lists for scores and thread identifiers.

#!/usr/bin/python
# coding=utf-8
from second import Violin as vv

class Fission:
    z = []
    x = []
    def __init__(self):
        print "欢迎使用fission类!"
    def getData(self, name):
        size = 0
        with open("/Users/Vicky/Documents/workspace/api_test/long/" + name + ".log") as apidata:
            for i in apidata:
                data = i.split("
")[0]
                time = float(data)
                if time > 2:
                    continue
                self.z.append(data)
                size += 1
        length = size
        for i in range(length):
            self.x.append(name + "线程")

if __name__ == "__main__":
    vvv = vv.Violin()
    fission = Fission()
    for i in range(10,23,1):
        aa = str((i + 1) * 10)
        fission.getData(aa)
    vvv.makeViolin(fission.z, fission.x, "接口性能测试", "110-230")

The second part encapsulates the chart generation logic in a Violin class that uses pandas and plotly.figure_factory to create a violin plot and export it as an HTML file.

#!/usr/bin/python
# coding=utf-8
import plotly.plotly
import pandas as pd
import plotly.figure_factory as ff

class Violin:
    def __init__(self):
        print "violin图表生成类！"
    def makeViolin(self, score, group, title, name):
        data = pd.DataFrame(dict(Score=score, Group=group))
        fig = ff.create_violin(data, data_header='Score', group_header='Group', height=700, width=1200, title=title)
        plotly.offline.plot(fig, filename=name + ".html")

Sample screenshots of the generated violin chart and QPS chart are included, and readers are invited to join a discussion group (ID: 340964272) for further exchange.