## New ElasticSearch storage option explanation in 9.2.0
Since v9.2.0, SkyWalking OAP provides 2 storage options for metrics/meter and records,
system environment variable is (`SW_STORAGE_ES_LOGIC_SHARDING`):
### No-Sharding Model (OAP default setting, `SW_STORAGE_ES_LOGIC_SHARDING = false`)
1. OAP merges all metrics/meter and records(without super datasets, such as segments) indices into one physical
index template `metrics-all` and `records-all`.
2. The logic index name would be present in columns `metric_table` and `record_table`.
3. If the logic column name has an alias (configured by `@ElasticSearch.Column()`), the alias would be the real physical column name.
### No-Sharding Model (`SW_STORAGE_ES_LOGIC_SHARDING = true `)
1. OAP shard metrics/meter indices into multi-physical indices as in the previous versions(one index template per metric/meter aggregation function).
2. Records and metrics without configuring aggregation function in `@MetricsFunction` and `@MeterFunction` would not be sharded.
3. The shard template name would be `metrics-aggregation function name` or `meter-aggregation function name` such as `metrics-count`,
and the logic index name would be present in column `metric_table`.
4. The OAP **would not** use the column alias, the logic column name would be the real physical column name.
**Notice**:
Users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out no matter the option.
* SQL Database: provide `@SQLDatabase.AdditionalEntity` to support create additional tables from a model.
* [Breaking Change] SQL Database: remove SQL Database config `maxSizeOfArrayColumn` and `numOfSearchableValuesPerTag`.
* [Breaking Change] SQL Database: move `Tags list` from `Segment`,`Logs`,`Alarms` to their additional table.
* Upgrade doc
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)
Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124
Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.
Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
* Fix application start duration log "Failed to load class "org.slf4j.impl.StaticLoggerBinder"." when use kafka reporter. because kafka reporter use slf4j-api, but no implementor for slf4j.
* Polishing.
* Adapter slf4j Logger of agent used to skywalking agent Logger.
* Make the comments more accurate.
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* #4463 FAQ for Maven compilation failure with error like python2 not found
* #4463 FAQ for Maven compilation failure with error like python2 not found
* Update maven-compile-npm-failure.md
* Update docs/en/FAQ/README.md
Co-Authored-By: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* Update docs/en/FAQ/maven-compile-npm-failure.md
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* Remove the local span and exit span register mechanism in Java agent scenario.
* Fix config, plugin, and document, as exit span is not required register anymore.
* Try to fix the test case expected data due to no register for exit span.
* Fix toolkit case.
* Remove exit span from plugin test framework document.
* Update protocol, FAQ, and CHANGELOG documents to highlight this change.
* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.
* Fix the snapshot and add inexistence operation name id.
* Fix several async cases.
* Fix undertow test case.
* Fix wrong error log name.
* Keep name consistent with 6.x concepts.
* Try to fix 3 async scenarios.
* Fix okhttp case.
* Fix Spring Async case.
* Fix another NPE
* Fix NPE when first span has no endpoint id when it is local or exit span.
* Still fill first operation name when no endpoint id.
吴晟 Wu Sheng
Wan Kai
cui fliter
Non-Official NPM Mirror Bot
kezhenxu94
Wing
John Bampton
zifeihan
Lu Jiajing
zifeihan
Daming
于玉桔
gongdewei
Gao Peng
mrproliu
Videl Smith
Jared Tan
yuyujulin
彭勇升 pengys
康智冬