Design and Implementation of a Cloud Audit Solution for Tencent Cloud Accounts

This article, authored by the Tencent Cloud Account and Permission team, presents a practical guide for building a cloud audit solution that gives administrators a "god view" of their Tencent Cloud resources. It combines security best practices with big‑data processing techniques.

Scenario : A customer (Client A) has a primary account with many sub‑accounts. The business wants to continuously monitor resource usage across all sub‑accounts, define custom audit rules, and receive timely alerts when those rules are triggered.

The authors propose a sustainable, extensible, and portable architecture that has been running in production for two months, handling terabytes of audit data.

Basic Concepts

• Cloud Audit : Captures API call history for a Tencent Cloud account, including calls from the console, SDKs, CLI, and other services. It records the caller, source IP, timestamp, and more.

• Trace Set : A configuration that forwards Cloud Audit events to a COS (Cloud Object Storage) bucket for persistent storage.

• COS : Distributed object storage service that can serve as a data lake for big‑data analysis.

• EMR : Elastic MapReduce service that provides managed Hadoop, Hive, Spark, etc., clusters for processing large datasets.

Detailed Solution

The solution consists of the following steps:

Store Cloud Audit logs in a COS bucket (must be in the Shanghai region).

Use EMR to run Hive queries on the COS data.

Create a COS‑backed Hive database and external table that maps Cloud Audit fields.

Schedule hourly jobs to add new partitions and load fresh data into Hive.

Aggregate cold data from COS into a hot store (e.g., MySQL) for fast querying and alerting.

Step 1: Connect to Hive

cd /usr/local/service/hive/bin && ./beeline -u "jdbc:hive2://10.0.0.1:7001" -n hadoop -p xxxxxx(你的Hive的密码)

Step 2: Create a COS‑backed database

create database cloudaudit location 'cosn://cloudaudit/xxx';

Step 3: Create the external Hive table

CREATE EXTERNAL TABLE `cloudaudit_logs`(
  `useridentity` struct
COMMENT '账户信息，其中accountid是主账号，principalid是当前操作账号',
  `eventtime` string COMMENT '事件发生的时间',
  `eventsource` string COMMENT '事件来源',
  `sourceipaddress` string COMMENT '事件发起的IP地址',
  `eventregion` string COMMENT '资源的地域',
  `eventname` string COMMENT '事件名称，例如：RestartInstances',
  `resourcetype` string COMMENT '资源类型，例如：CVM',
  `useragent` string COMMENT '调用方Agent',
  `errorcode` string COMMENT '错误码，0代表正确',
  `errormessage` string COMMENT '错误描述',
  `additionaleventdata` struct
COMMENT '登录才有的额外字段',
  `requestid` string COMMENT '请求的唯一ID',
  `eventid` string COMMENT '日志的唯一ID',
  `eventtype` string COMMENT '请求类型，ConsoleCall代表控制台操作，ApiCall代表API调用，ConsoleLogin代表登录',
  `apiversion` string COMMENT 'api的版本',
  `actiontype` string COMMENT '动作类型',
  `resources` array
COMMENT '资源',
  `resourcename` string COMMENT '资源名称'
) PARTITIONED BY (`dt` string) -- hourly partition
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES ('ignore.malformed.json'='true')
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 'cosn://cloudaudit/xxxx';

Step 4: Hourly partition management (Python)

lastHour = (datetime.datetime.now() - datetime.timedelta(hours=1)).strftime('%Y%m%d%H')

hive_client = hiveClient.HiveClient(
    db_host=config.HIVE_CONFIG['host'],
    port=config.HIVE_CONFIG['port'],
    user=config.HIVE_CONFIG['user'],
    password=config.HIVE_CONFIG['passWord'],
    database=config.HIVE_CONFIG['database'],
    authMechanism=config.HIVE_CONFIG['authMechanism']
)

# Check if partition exists, create if not
partitionSql = 'show partitions cloudaudit_logs'
result = hive_client.query(partitionSql)
partitions = []
for row in result:
    partitions.append(row[0])
if 'dt=' + lastHour not in partitions:
    alterPartitionSql = 'ALTER TABLE cloudaudit_logs ADD PARTITION (dt="' + lastHour + '") location "cosn://cloudaudit/xxxx/' + lastHour + '"'
    hive_client.queryWithoutReturn(alterPartitionSql)

Step 5: Aggregate cold data into hot storage (Python)

# Execute aggregation query
sql = 'select useridentity.accountid, userIdentity.principalId, sourceIPAddress, eventName, resourceType, errorCode, eventType, count(*) as cnt '
sql += 'from cloudaudit_logs where dt = "' + lastHour + '" '
sql += 'group by userIdentity.accountid, sourceIPAddress, userIdentity.principalId, eventName, resourceType, errorCode, eventType'
result = hive_client.query(sql)

hive_client.close()

After aggregation, the results are written to MySQL (or another hot store) and can be used to implement custom audit rules and dashboards that give the “god view” of the entire account.

The article also lists several practical constraints, such as the requirement that COS buckets be created in Shanghai, EMR clusters should also be in Shanghai to reduce cross‑region traffic, and that only control‑flow logs are currently supported.

Illustrations (omitted here) show the overall architecture, operation timelines, and anomaly region visualizations.