chore(logger): upgrade logforth to support global max_files limit for log_dir

[bohutang]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Upgrade logforth with Global File Limit Capability

Upgrades to logforth from https://github.com/datafuse-extras/logforth (global-max-files-v0.14 branch), which adds the ability to limit the total number of log files in log_dir via PR #1.

• fixes: chore: enable limit total number of log files #17635

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - tests already in datafuse-extras/logforth

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe): enhance the log files limitition

---

This change is 

[Xuanwo]

This is a temporary solution, right? I see logforth 0.23 has already supported the max file limit: https://github.com/fast/logforth/blob/d9620447ba9be53a730d8960f103445de79076a9/src/append/rolling_file/rolling.rs#L137-L142

[bohutang]

This is a temporary solution, right? I see logforth 0.23 has already supported the max file limit: https://github.com/fast/logforth/blob/d9620447ba9be53a730d8960f103445de79076a9/src/append/rolling_file/rolling.rs#L137-L142

No, this is the final solution.
The max_files is for rotation(if hourly the max_files mean not over max_files in a hour) not global all in log_dir, see : fast/logforth#107
BTW, it's hard to use the latest version of logforth, there are many api need to change, so we don't have the plan to upgrade to the latest version.

[wubx]

The log section with:

[log.file]
limit = 12
max-size=1024
format = "text"
include_node_id=true
dir = "/var/log/databend"
level = "WARN,databend=info,opendal=info,openraft=info"

Test use

for i in `seq 1 1000` ; do echo "select $i"|bendsql ; done

"It feels like the number of generated log files is far beyond what was expected."

[bohutang]

cc @wubx

From my local test, this works as expected.

[log.file]
limit = 3
max-size=256
format = "text"
include_node_id=true
dir = "./logs"
level = "INFO"

I only tested as:
Case 1:
The logs have many log files(64) and start the query, the log files reduce to 3.

Case 2:
Do many times like select sum(numbers) from numbers(10000), the logs directory always keep 3 log files.

[tisonkun]

From the latest code I don't see the date time being used as part of filename (prefix), then we may not need this workaround now?

See fast/logforth#207.