Master jsrpc: Bypass Encryption and Automate Python Web Scraping in 15 Steps

In this article the author shares a practical jsrpc tutorial that demonstrates how to reverse‑engineer a website where the cookie remains constant but the encrypted Sign parameter changes, and then use the extracted signature to automate data collection.

Step‑by‑step process

1. Identify the target URL, which is an MD5‑hashed endpoint 87aed0b6bc8cb687d63dd7eee0f64d38.

2. Extract numeric values from 100 pages and compute their sum.

3. Open the browser’s network panel, set breakpoints, and locate the encryption function that generates the request parameters.

4. Find the encryption function in the JavaScript code and expose it via the console: window.dcpeng = window.get_sign Note: assigning the function itself (without parentheses) is crucial; using ct.update() would assign the result instead.

5. Disable the page’s debug mode to allow WebSocket injection.

6. Run the compiled win64-localhost.exe to start the local jsrpc service.

7. Inject the JavaScript environment by copying the entire content of JsEnv.js into the browser console.

8. Connect to the service with:

var demo = new Hlclient("ws://127.0.0.1:12080/ws?group=para&name=test");

9. Register a method that retrieves the signature:

// Register a method named get_para
demo.regAction("get_para", function (resolve) {
    dcpeng();
    var res = window.sign;
    resolve(res);
});

10. Call the registered method via HTTP:

http://127.0.0.1:12080/go?group=para&name=test&action=get_para

11. Use the returned v value in subsequent requests. The following Python script demonstrates the full crawling loop:

import requests
import json

cookies = {
    'session': '6c78df1c-37aa-4574-bb50-99784ffb3697.Qcl0XN6livMeZ-7tbiNe-Ogn8L4',
    'v': 'A7s8gqX6XgjWtmKFwCNKPNdQSpQgEM9-ySWTzq14lzDRLtVKNeBfYtn0IxW-'
}

headers = {
    'Connection': 'keep-alive',
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'X-Requested-With': 'XMLHttpRequest',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36 Edg/97.0.1072.69',
    'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
    'Origin': 'http://spider.wangluozhe.com',
    'Referer': 'http://spider.wangluozhe.com/challenge/2',
    'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6'
}

all_data = []
for page_num in range(1, 101):
    sign_url = 'http://127.0.0.1:12080/go?group=para&name=test&action=get_para'
    sign = requests.get(url=sign_url).json()["get_para"]
    data = {
        'page': f'{page_num}',
        'count': '10',
        '_signature': sign
    }
    print(f'Crawlering page {page_num}')
    response = requests.post('87aed0b6bc8cb687d63dd7eee0f64d38', headers=headers, cookies=cookies, data=data, verify=False).json()
    for item in response["data"]:
        all_data.append(item["value"])

print(sum(all_data))

The script prints the sum of all extracted values, matching the data shown on the target website.

By following these steps you can fully automate the data extraction process for the challenged site, and the same approach can be adapted to similar anti‑scraping mechanisms.

Conclusion

The jsrpc tool enables a black‑box style reverse engineering workflow that eliminates the need for manual environment setup, allowing rapid acquisition of protected data through a simple RPC interface.