Pandas Data Objects: Series, DataFrame Creation, Indexing, CRUD Operations, and Common Functions
This tutorial introduces pandas' two core data objects—Series and DataFrame—demonstrates how to create, index, query, modify, add, delete, sort, merge, and copy them, and shows common parameters, functions, I/O operations, plotting, and a practical log‑analysis example using Python.
This article provides a comprehensive guide to pandas, a Python library for data manipulation, focusing on its two primary data structures: Series and DataFrame .
Series is a one‑dimensional labeled array. A simple creation example is:
# 通过传入一个序列给pd.Series初始化一个Series对象, 比如lists1 = pd.Series(list("1234"))print(s1)0 11 22 33 4dtype: objectDataFrame is a two‑dimensional table‑like structure. It can be created from a NumPy array or a dictionary:
# 通过传入一个numpy的二维数组或者dict对象给pd.DataFrame初始化一个DataFrame对象import numpy as npdf1 = pd.DataFrame(np.random.randn(6,4))print(df1)# 通过dict字典df2 = pd.DataFrame({
'A' : 1.,
'B' : pd.Timestamp('20130102'),
'C' : pd.Series(1, index=list(range(4)), dtype='float32'),
'D' : np.array([3] * 4, dtype='int32'),
'E' : pd.Categorical(["test","train","test","train"]),
'F' : 'foo'
})print(df2)Both objects have an index; for Series the index labels each element, while for DataFrame it labels rows. The index can be inspected with print(s1.index) or print(df.index).
Selection and Query – you can view columns, rows, or subsets using .head(), .tail(), slicing, .loc (label‑based) and .iloc (position‑based). Examples:
# 查看前5行df.head()# 查看第10行的date和code列df.loc[10, ["date", "code"]]# 按位置查看第1行df.iloc[0]Conditional filtering uses boolean masks:
# 查看open列大于10的前5行df[df.open > 10].head()# 同时满足两个条件df[(df.open > 10) & (df.open < 10.6)].head()Modification – change index, rename columns, or apply functions:
# 将df的索引修改为date列的数据并转换为datetimedf.index = pd.to_datetime(df.date)# 重命名列df.columns = ["Date","Open","Close","High","Low","Volume","Code"]# 给Open列每个数值加1df.Open = df.Open.apply(lambda x: x+1)Deletion – use .drop() with inplace=True if you want to modify the original object:
# 删除Open列df.drop("Open", axis=1, inplace=True)Common parameters and functions – control display format, obtain descriptive statistics, and handle missing values:
# 设置浮点数显示格式pd.options.display.float_format = '{:.4f}'.format# 统计描述df.describe()# 删除含NaN的行df.dropna(how='any')# 用固定值填充NaNdf.fillna(value=5)Sorting and Merging – sort by index or values, concatenate rows/columns, or merge datasets:
# 按列排序df.sort_index(axis=1).head()# 按Open列升序排序df.sort_values(by="Open").head()# 行方向合并pd.concat([df.iloc[0:2,:], df.iloc[2:4,:], df.iloc[4:9]])Copying – because DataFrames are mutable references, use .copy() to create an independent copy before making changes.
Plotting – pandas integrates with matplotlib. Simple line and area plots can be generated with:
df[["Open","Low","High","Close"]].plot()df[["Open","Low","High","Close"]].plot(kind="area")Data I/O – read and write common file formats such as CSV, Excel, JSON, or even the system clipboard:
# 保存为CSVdf.to_csv("stock.csv")# 从CSV读取并指定第一列为索引df2 = pd.read_csv("stock.csv", index_col=0)Practical Example: Web Log Analysis – the article demonstrates parsing an Apache access log into a DataFrame, converting the timestamp, casting numeric fields, and performing analyses such as status‑code counts, top IP addresses, and time‑series visualisations.
# 正则表达式解析日志REGEX = HOST+SPACE+IDENTITY+SPACE+USER+SPACE+TIME+SPACE+REQUEST+SPACE+STATUS+SPACE+SIZE+SPACE+IDENTITY+USER_AGENT+SPACElog_df = pd.DataFrame(field_lis, columns=["Host","Time","Method","Path","Protocol","status","size","User_Agent"])# 将Time列转为datetime并设为索引log_df.Time = log_df.Time.apply(lambda x: x.replace(":", " ", 1))
log_df.Time = pd.to_datetime(log_df.Time)
log_df.set_index('Time', inplace=True)# 统计不同状态码出现次数log_df.Status.value_counts()# 绘制状态码饼图log_df.Status.value_counts().plot(kind="pie", figsize=(10,8))Overall, the guide equips readers with the essential pandas operations needed for data cleaning, transformation, analysis, and visualization, making it a solid reference for anyone working with tabular data in Python.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
