Generate Tang Poetry with Python: Scraping, Processing, and Rhyme Creation

It is often said that modern code is as important as Tang poetry, and this article shows how to actually write a Tang poem using Python.

Preparation

Python 3.6 environment

Recommended: Anaconda for package management and isolated environments

Recommended IDE: PyCharm

GitHub repository: https://github.com/theodore3131/TangshiGenerator

Specific Steps

1. Use a web crawler to fetch the complete Tang poetry corpus (about 71,000 poems).

# Build a safe crawler using urllib3's built‑in verification
http = urllib3.PoolManager(
    cert_reqs='CERT_REQUIRED',
    ca_certs=certifi.where()
)
# Target website
r = http.request('GET', url)
# Parse the HTML response
soup = BeautifulSoup(r.data, 'html.parser')
content = soup.find('div', class_="contson")

2. Process the fetched data with regular expressions.

p1 = r"[一-龥]{5,7}[。|，]"  # Match 5‑7 Chinese characters followed by a period or comma
pattern1 = re.compile(p1)
result = pattern1.findall(poemfile)  # Get a list of matching strings

3. Tokenize the poem text using Jieba's TextRank algorithm to extract high‑frequency words.

for x in jieba.analyse.textrank(content, topK=600,
    allowPOS=('n','nr','ns','nt','nz','m')):
    pass

4. Generate verses and handle rhymes with the pinyin library.

# Install the pinyin library
pip install pinyin
verse = pinyin.get("天", format="strip")  # Output: tian

Rhyme detection relies on the fact that all rhyme endings start with a vowel in the pinyin representation. The algorithm extracts the last vowel sequence as the rhyme.

rhythm = ""
rhythmList = ["a","e","i","o","u"]
verse = pinyin.get(nounlist[i1][1], format="strip")
for p in range(len(verse)-1, -1, -1):
    if verse[p] in rhythmList:
        ind = p
        break
rhythm = verse[ind:]

5. Assemble a basic five‑character regulated poem (五言律诗) using random selection while ensuring that the second and fourth lines share the same rhyme.

while num < 4:
    i = random.randint(1, len(nounlist)-1)
    i1 = random.randint(1, len(nounlist)-1)
    j = random.randint(1, len(verblist)-1)
    # Determine rhyme for line 2 and 4
    if num == 1:
        verse = pinyin.get(nounlist[i1][1], format="strip")
        # extract rhyme as above
    if num == 3:
        # ensure rhyme matches line 2
        while verse1[ind1:] != rhythm:
            i1 = random.randint(1, len(nounlist)-1)
            verse1 = pinyin.get(nounlist[i1][1], format="strip")
            # extract rhyme again
    print(nounlist[i] + verblist[j][1] + nounlist[i1])
    num += 1

Acrostic Poem (藏头诗)

The idea is simple: when assembling each line, ensure that the first character of the first noun follows a predefined four‑character idiom sequence.

for x in range(len(nounlist)):
    if nounlist[x][0] == str[num]:
        i = x

Example results:

Four‑character poem:

所思浮云
关山车马
高楼流水
闲人肠断

Five‑character regulated poem:

西风时细雨
山川钓建章
龙门看萧索
几年乡斜阳

Acrostic poem:

落花流水
落晖首南宫
花枝成公子
流水名朝廷
水声胜白石

Currently the generated poems are basic and rely on simple word permutations. Future work includes extracting common five‑ and seven‑character sentence templates and applying machine‑learning models such as RNNs to produce more nuanced and aesthetically pleasing poetry.