Master Converting String Dates to Datetime in Pandas with pd.to_datetime

Data Requirement

The task is to convert a column containing date values represented as strings, such as 2021-11-27 00:00:00, into proper datetime types.

Simple Solution

In most cases you can directly apply pd.to_datetime(df['date']) without specifying a format; pandas will infer the appropriate datetime type automatically.

import pandas as pd

df = ...

# Show original string values
print(df['date'].values)

# Convert to datetime
converted = pd.to_datetime(df['date'])
print(converted)

Key Parameters of pd.to_datetime

errors : Determines how conversion errors are handled. Options are raise (default, stops execution), coerce (invalid parsing becomes pd.NaT), and ignore (returns the original input).

format : Explicit date format string (e.g., %Y-%m-%d) used when pandas cannot infer the format automatically.

infer_datetime_format : When set to True, pandas uses the first successfully parsed format as a template for the rest, often speeding up conversion by 5‑10× for consistent formats.

unit : Specifies the time unit of integer timestamps (default ns). Common units are ms, s, and D.

origin : Reference point for numeric timestamps. The default unix corresponds to 1970‑01‑01 00:00:00 UTC; a custom origin can be provided as a timestamp.

Examples of Various Formats

pd.to_datetime(['03Apr2022', '04-Apr-2022', '2022-01-01', '20220101', '2022/01/01', 'Oct 10 2021'])

When the format is ambiguous, you can supply a format string:

# Day‑Month‑Year
pd.to_datetime(['010122'], format='%d%m%y')

# Month‑Day‑Year
pd.to_datetime(['020122'], format='%m%d%y')

# Year‑Month‑Day (two‑digit year)
pd.to_datetime(['220301'], format='%y%m%d')

Timestamp Conversion

# Milliseconds
pd.to_datetime([1648195805123], unit='ms')

# Seconds
pd.to_datetime([1648195805], unit='s')

# Days since epoch
pd.to_datetime([19076], unit='D')

Custom origins adjust the base date for numeric timestamps:

pd.to_datetime([1, 2, 3], unit='D', origin=pd.Timestamp('1960-01-01'))

Summary

Understanding and using pd.to_datetime simplifies date handling in pandas, leveraging the underlying datetime library while offering flexible parameters for error handling, format specification, performance optimization, and timestamp conversion, making data cleaning more robust and efficient.