kazusa
/
skywalking
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Unified the config word `namespace` in the project (#7676)

This commit is contained in:
wankai123 2021-09-08 12:00:19 +08:00 committed by GitHub
parent 6007f3ce2d
commit 4a344b01e9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
20 changed files with 51 additions and 50 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGES.md
View File

@ -58,6 +58,7 @@ Release Notes.
* Rebuilt ElasticSearch client on top of their REST API.
* Fix ElasticSearch storage plugin doesn't work when hot reloading from `secretsManagementFile`.
* Support etcd grouped dynamic configurations.
* Unified the config word `namespace` in the project.
#### UI

2
docs/en/setup/backend/backend-cluster.md
View File

@ -47,7 +47,7 @@ The following settings are provided to set the host and port manually, based on
```yaml
zookeeper:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
hostPort: ${SW_CLUSTER_ZK_HOST_PORT:localhost:2181}
#Retry Policy
baseSleepTimeMs: ${SW_CLUSTER_ZK_SLEEP_TIME:1000} # initial amount of time to wait between retries

6
docs/en/setup/backend/backend-storage.md
View File

@ -54,7 +54,7 @@ follows:
storage:
selector: ${SW_STORAGE:elasticsearch}
elasticsearch:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
trustStorePath: ${SW_STORAGE_ES_SSL_JKS_PATH:""}
@ -90,7 +90,7 @@ Example:
storage:
selector: ${SW_STORAGE:elasticsearch}
elasticsearch:
# nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
user: ${SW_ES_USER:""} # User needs to be set when Http Basic authentication is enabled
password: ${SW_ES_PASSWORD:""} # Password to be set when Http Basic authentication is enabled
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:443}
@ -166,7 +166,7 @@ The configurations are largely the same.
storage:
selector: ${SW_STORAGE:zipkin-elasticsearch}
zipkin-elasticsearch:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
user: ${SW_ES_USER:""}

6
docs/en/setup/backend/configuration-vocabulary.md
View File

@ -47,7 +47,7 @@ core|default|role|Option values: `Mixed/Receiver/Aggregator`. **Receiver** mode
| - | - | prepareThreads| The number of threads used to prepare metrics data to the storage. | SW_CORE_PREPARE_THREADS | 2 |
| - | - | enableEndpointNameGroupingByOpenapi | Automatically groups endpoints by the given OpenAPI definitions. | SW_CORE_ENABLE_ENDPOINT_NAME_GROUPING_BY_OPAENAPI | true |
|cluster|standalone| - | Standalone is not suitable for running on a single node running. No configuration available. | - | - |
| - | zookeeper|nameSpace| The namespace, represented by root path, isolates the configurations in Zookeeper.|SW_NAMESPACE| `/`, root path|
| - | zookeeper|namespace| The namespace, represented by root path, isolates the configurations in Zookeeper.|SW_NAMESPACE| `/`, root path|
| - | - | hostPort| Hosts and ports of Zookeeper Cluster. |SW_CLUSTER_ZK_HOST_PORT| localhost:2181|
| - | - | baseSleepTimeMs| The period of Zookeeper client between two retries (in milliseconds). |SW_CLUSTER_ZK_SLEEP_TIME|1000|
| - | - | maxRetries| The maximum retry time. |SW_CLUSTER_ZK_MAX_RETRIES|3|
@ -80,7 +80,7 @@ core|default|role|Option values: `Mixed/Receiver/Aggregator`. **Receiver** mode
| - | - | accessKey | Nacos Auth accessKey. | SW_CLUSTER_NACOS_ACCESSKEY | - |
| - | - | secretKey | Nacos Auth secretKey. | SW_CLUSTER_NACOS_SECRETKEY | - |
| storage|elasticsearch| - | ElasticSearch (and OpenSearch) storage implementation. | - | - |
| - | - | nameSpace | Prefix of indexes created and used by SkyWalking. | SW_NAMESPACE | - |
| - | - | namespace | Prefix of indexes created and used by SkyWalking. | SW_NAMESPACE | - |
| - | - | clusterNodes | ElasticSearch cluster nodes for client connection.| SW_STORAGE_ES_CLUSTER_NODES |localhost|
| - | - | protocol | HTTP or HTTPs. | SW_STORAGE_ES_HTTP_PROTOCOL | HTTP|
| - | - | connectTimeout | Connect timeout of ElasticSearch client (in milliseconds). | SW_STORAGE_ES_CONNECT_TIMEOUT | 500|
@ -236,7 +236,7 @@ core|default|role|Option values: `Mixed/Receiver/Aggregator`. **Receiver** mode
| - | - | apolloEnv | `env` in Apollo. | SW_CONFIG_APOLLO_ENV | - |
| - | - | appId | `app.id` in Apollo. | SW_CONFIG_APOLLO_APP_ID | skywalking |
| - | - | period | The period of data sync (in seconds). | SW_CONFIG_APOLLO_PERIOD | 60 |
| - | zookeeper|nameSpace| The namespace (represented by root path) that isolates the configurations in the Zookeeper. |SW_CONFIG_ZK_NAMESPACE| `/`, root path|
| - | zookeeper|namespace| The namespace (represented by root path) that isolates the configurations in the Zookeeper. |SW_CONFIG_ZK_NAMESPACE| `/`, root path|
| - | - | hostPort| Hosts and ports of Zookeeper Cluster. |SW_CONFIG_ZK_HOST_PORT| localhost:2181|
| - | - | baseSleepTimeMs|The period of Zookeeper client between two retries (in milliseconds). |SW_CONFIG_ZK_BASE_SLEEP_TIME_MS|1000|
| - | - | maxRetries| The maximum retry time. |SW_CONFIG_ZK_MAX_RETRIES|3|

12
docs/en/setup/backend/dynamic-config-zookeeper.md
View File

@ -6,26 +6,26 @@ configuration:
selector: ${SW_CONFIGURATION:zookeeper}
zookeeper:
period: ${SW_CONFIG_ZK_PERIOD:60} # Unit seconds, sync period. Default fetch every 60 seconds.
nameSpace: ${SW_CONFIG_ZK_NAMESPACE:/default}
namespace: ${SW_CONFIG_ZK_NAMESPACE:/default}
hostPort: ${SW_CONFIG_ZK_HOST_PORT:localhost:2181}
# Retry Policy
baseSleepTimeMs: ${SW_CONFIG_ZK_BASE_SLEEP_TIME_MS:1000} # initial amount of time to wait between retries
maxRetries: ${SW_CONFIG_ZK_MAX_RETRIES:3} # max number of times to retry
```
The **nameSpace** is the ZooKeeper path. The config key and value are the properties of the `namespace` folder.
The **namespace** is the ZooKeeper path. The config key and value are the properties of the `namespace` folder.
## Config Storage
### Single Config
```
znode.path = {nameSpace}/configKey
znode.path = {namespace}/configKey
configValue = znode.data
```
e.g. The config is:
```
{agent-analyzer.default.slowDBAccessThreshold}:{default:200,mongodb:50}
```
If `nameSpace = /default` the config in zookeeper is:
If `namespace = /default` the config in zookeeper is:
```
znode.path = /default/agent-analyzer.default.slowDBAccessThreshold
znode.data = default:200,mongodb:50
@ -33,7 +33,7 @@ znode.data = default:200,mongodb:50
### Group Config
```
znode.path = {nameSpace}/configKey
znode.path = {namespace}/configKey
znode.child1.path = {znode.path}/subItemkey1
znode.child2.path = {znode.path}/subItemkey2
...
@ -47,7 +47,7 @@ e.g. The config is:
|{productAPI-v1}:{value of productAPI-v1}
|{productAPI-v2}:{value of productAPI-v2}
```
If `nameSpace = /default` the config in zookeeper is:
If `namespace = /default` the config in zookeeper is:
```
znode.path = /default/core.default.endpoint-name-grouping-openapi
znode.customerAPI-v1.path = /default/core.default.endpoint-name-grouping-openapi/customerAPI-v1

2
oap-server/server-cluster-plugin/cluster-zookeeper-plugin/src/main/java/org/apache/skywalking/oap/server/cluster/plugin/zookeeper/ClusterModuleZookeeperConfig.java
View File

@ -27,7 +27,7 @@ class ClusterModuleZookeeperConfig extends ModuleConfig {
@Setter
@Getter
private String nameSpace;
private String namespace;
private String hostPort;
private int baseSleepTimeMs;
private int maxRetries;

2
oap-server/server-cluster-plugin/cluster-zookeeper-plugin/src/main/java/org/apache/skywalking/oap/server/cluster/plugin/zookeeper/ClusterModuleZookeeperProvider.java
View File

@ -118,7 +118,7 @@ public class ClusterModuleZookeeperProvider extends ModuleProvider {
}
client = builder.build();
String path = BASE_PATH + (StringUtil.isEmpty(config.getNameSpace()) ? "" : "/" + config.getNameSpace());
String path = BASE_PATH + (StringUtil.isEmpty(config.getNamespace()) ? "" : "/" + config.getNamespace());
serviceDiscovery = ServiceDiscoveryBuilder.builder(RemoteInstance.class)
.client(client)

2
oap-server/server-cluster-plugin/cluster-zookeeper-plugin/src/test/java/org/apache/skywalking/oap/server/cluster/plugin/zookeeper/ITClusterModuleZookeeperProviderFunctionalTest.java
View File

@ -199,7 +199,7 @@ public class ITClusterModuleZookeeperProviderFunctionalTest {
moduleConfig.setMaxRetries(3);
if (!StringUtil.isEmpty(namespace)) {
moduleConfig.setNameSpace(namespace);
moduleConfig.setNamespace(namespace);
}
if (!StringUtil.isEmpty(internalComHost)) {

4
oap-server/server-configuration/configuration-zookeeper/src/main/java/org/apache/skywalking/oap/server/configuration/zookeeper/ZookeeperConfigWatcherRegister.java
View File

@ -39,11 +39,11 @@ public class ZookeeperConfigWatcherRegister extends ConfigWatcherRegister {
public ZookeeperConfigWatcherRegister(ZookeeperServerSettings settings) throws Exception {
super(settings.getPeriod());
prefix = settings.getNameSpace() + "/";
prefix = settings.getNamespace() + "/";
RetryPolicy retryPolicy = new ExponentialBackoffRetry(settings.getBaseSleepTimeMs(), settings.getMaxRetries());
this.client = CuratorFrameworkFactory.newClient(settings.getHostPort(), retryPolicy);
client.start();
this.childrenCache = new PathChildrenCache(client, settings.getNameSpace(), true);
this.childrenCache = new PathChildrenCache(client, settings.getNamespace(), true);
this.childrenCache.start();
}

4
oap-server/server-configuration/configuration-zookeeper/src/main/java/org/apache/skywalking/oap/server/configuration/zookeeper/ZookeeperConfigurationProvider.java
View File

@ -49,8 +49,8 @@ public class ZookeeperConfigurationProvider extends AbstractConfigurationProvide
if (Strings.isNullOrEmpty(settings.getHostPort())) {
throw new ModuleStartException("Zookeeper hostPort cannot be null or empty.");
}
if (Strings.isNullOrEmpty(settings.getNameSpace())) {
throw new ModuleStartException("Zookeeper nameSpace cannot be null or empty.");
if (Strings.isNullOrEmpty(settings.getNamespace())) {
throw new ModuleStartException("Zookeeper namespace cannot be null or empty.");
}
try {

2
oap-server/server-configuration/configuration-zookeeper/src/main/java/org/apache/skywalking/oap/server/configuration/zookeeper/ZookeeperServerSettings.java
View File

@ -27,7 +27,7 @@ import org.apache.skywalking.oap.server.library.module.ModuleConfig;
@Setter
@ToString
public class ZookeeperServerSettings extends ModuleConfig {
private String nameSpace = "/default";
private String namespace = "/default";
private String hostPort;
private int baseSleepTimeMs = 1000;
private int maxRetries = 3;

26
oap-server/server-configuration/configuration-zookeeper/src/test/java/org/apache/skywalking/oap/server/configuration/zookeeper/it/ITZookeeperConfigurationTest.java
View File

@ -78,23 +78,23 @@ public class ITZookeeperConfigurationTest {
@SuppressWarnings("StatementWithEmptyBody")
@Test(timeout = 20000)
public void shouldReadUpdated() throws Exception {
String nameSpace = "/default";
String namespace = "/default";
String key = "test-module.default.testKey";
assertNull(provider.watcher.value());
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
CuratorFramework client = CuratorFrameworkFactory.newClient(zkAddress, retryPolicy);
client.start();
log.info("per path: " + nameSpace + "/" + key);
log.info("per path: " + namespace + "/" + key);
assertTrue(client.create().creatingParentsIfNeeded().forPath(nameSpace + "/" + key, "500".getBytes()) != null);
assertTrue(client.create().creatingParentsIfNeeded().forPath(namespace + "/" + key, "500".getBytes()) != null);
log.info("data: " + new String(client.getData().forPath(nameSpace + "/" + key)));
log.info("data: " + new String(client.getData().forPath(namespace + "/" + key)));
for (String v = provider.watcher.value(); v == null; v = provider.watcher.value()) {
}
assertTrue(client.delete().forPath(nameSpace + "/" + key) == null);
assertTrue(client.delete().forPath(namespace + "/" + key) == null);
for (String v = provider.watcher.value(); v != null; v = provider.watcher.value()) {
}
@ -104,28 +104,28 @@ public class ITZookeeperConfigurationTest {
@Test(timeout = 20000)
public void shouldReadUpdated4GroupConfig() throws Exception {
String nameSpace = "/default";
String namespace = "/default";
String key = "test-module.default.testKeyGroup";
assertEquals("{}", provider.groupWatcher.groupItems().toString());
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
CuratorFramework client = CuratorFrameworkFactory.newClient(zkAddress, retryPolicy);
client.start();
log.info("per path: " + nameSpace + "/" + key);
log.info("per path: " + namespace + "/" + key);
assertTrue(client.create().creatingParentsIfNeeded().forPath(nameSpace + "/" + key + "/item1", "100".getBytes()) != null);
assertTrue(client.create().creatingParentsIfNeeded().forPath(nameSpace + "/" + key + "/item2", "200".getBytes()) != null);
assertTrue(client.create().creatingParentsIfNeeded().forPath(namespace + "/" + key + "/item1", "100".getBytes()) != null);
assertTrue(client.create().creatingParentsIfNeeded().forPath(namespace + "/" + key + "/item2", "200".getBytes()) != null);
log.info("data: " + new String(client.getData().forPath(nameSpace + "/" + key + "/item1")));
log.info("data: " + new String(client.getData().forPath(nameSpace + "/" + key + "/item2")));
log.info("data: " + new String(client.getData().forPath(namespace + "/" + key + "/item1")));
log.info("data: " + new String(client.getData().forPath(namespace + "/" + key + "/item2")));
for (String v = provider.groupWatcher.groupItems().get("item1"); v == null; v = provider.groupWatcher.groupItems().get("item1")) {
}
for (String v = provider.groupWatcher.groupItems().get("item2"); v == null; v = provider.groupWatcher.groupItems().get("item2")) {
}
assertTrue(client.delete().forPath(nameSpace + "/" + key + "/item1") == null);
assertTrue(client.delete().forPath(nameSpace + "/" + key + "/item2") == null);
assertTrue(client.delete().forPath(namespace + "/" + key + "/item1") == null);
assertTrue(client.delete().forPath(namespace + "/" + key + "/item2") == null);
for (String v = provider.groupWatcher.groupItems().get("item1"); v != null; v = provider.groupWatcher.groupItems().get("item1")) {
}

2
oap-server/server-configuration/configuration-zookeeper/src/test/java/org/apache/skywalking/oap/server/configuration/zookeeper/ut/MockZookeeperConfigWatcherRegister.java
View File

@ -33,7 +33,7 @@ public class MockZookeeperConfigWatcherRegister extends ConfigWatcherRegister {
public MockZookeeperConfigWatcherRegister(ZookeeperServerSettings settings) throws Exception {
super(settings.getPeriod());
prefix = settings.getNameSpace() + "/";
prefix = settings.getNamespace() + "/";
}
@Override

6
oap-server/server-configuration/configuration-zookeeper/src/test/java/org/apache/skywalking/oap/server/configuration/zookeeper/ut/ZookeeperConfigWatcherRegisterTestCase.java
View File

@ -34,16 +34,16 @@ import static org.mockito.Mockito.when;
public class ZookeeperConfigWatcherRegisterTestCase {
@Test
public void TestCase() throws Exception {
final String nameSpace = "/default";
final String namespace = "/default";
final String key = "receiver-trace.default.slowDBAccessThreshold";
final String value = "default:100,mongodb:50";
final ZookeeperServerSettings mockSettings = mock(ZookeeperServerSettings.class);
when(mockSettings.getNameSpace()).thenReturn(nameSpace);
when(mockSettings.getNamespace()).thenReturn(namespace);
final MockZookeeperConfigWatcherRegister mockRegister = spy(new MockZookeeperConfigWatcherRegister(mockSettings));
final PathChildrenCache mockPathChildrenCache = mock(PathChildrenCache.class);
when(mockPathChildrenCache.getCurrentData(nameSpace + "/" + key)).thenReturn(new ChildData(nameSpace + "/" + key, null, value
when(mockPathChildrenCache.getCurrentData(namespace + "/" + key)).thenReturn(new ChildData(namespace + "/" + key, null, value
.getBytes()));
Whitebox.setInternalState(mockRegister, "childrenCache", mockPathChildrenCache);

2
oap-server/server-configuration/configuration-zookeeper/src/test/resources/application.yml
View File

@ -20,7 +20,7 @@ test-module:
configuration:
zookeeper:
period : 1 # Unit seconds, sync period. Default fetch every 60 seconds.
nameSpace: /default
namespace: /default
hostPort: ${zk.address}
baseSleepTimeMs: 1000 # initial amount of time to wait between retries
maxRetries: 3 # max number of times to retry

2
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleConfig.java
View File

@ -28,7 +28,7 @@ import org.apache.skywalking.oap.server.library.module.ModuleConfig;
@Getter
public class CoreModuleConfig extends ModuleConfig {
private String role = "Mixed";
private String nameSpace;
private String namespace;
private String restHost;
private int restPort;
private String restContextPath;

8
oap-server/server-starter/src/main/resources/application.yml
View File

@ -19,7 +19,7 @@ cluster:
# Please check your ZooKeeper is 3.5+, However, it is also compatible with ZooKeeper 3.4.x. Replace the ZooKeeper 3.5+
# library the oap-libs folder with your ZooKeeper 3.4.x library.
zookeeper:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
hostPort: ${SW_CLUSTER_ZK_HOST_PORT:localhost:2181}
# Retry Policy
baseSleepTimeMs: ${SW_CLUSTER_ZK_SLEEP_TIME:1000} # initial amount of time to wait between retries
@ -120,7 +120,7 @@ core:
storage:
selector: ${SW_STORAGE:h2}
elasticsearch:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
connectTimeout: ${SW_STORAGE_ES_CONNECT_TIMEOUT:500}
@ -207,7 +207,7 @@ storage:
maxSizeOfArrayColumn: ${SW_STORAGE_MAX_SIZE_OF_ARRAY_COLUMN:20}
numOfSearchableValuesPerTag: ${SW_STORAGE_NUM_OF_SEARCHABLE_VALUES_PER_TAG:2}
zipkin-elasticsearch:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
trustStorePath: ${SW_STORAGE_ES_SSL_JKS_PATH:""}
@ -425,7 +425,7 @@ configuration:
period: ${SW_CONFIG_APOLLO_PERIOD:5}
zookeeper:
period: ${SW_CONFIG_ZK_PERIOD:60} # Unit seconds, sync period. Default fetch every 60 seconds.
nameSpace: ${SW_CONFIG_ZK_NAMESPACE:/default}
namespace: ${SW_CONFIG_ZK_NAMESPACE:/default}
hostPort: ${SW_CONFIG_ZK_HOST_PORT:localhost:2181}
# Retry Policy
baseSleepTimeMs: ${SW_CONFIG_ZK_BASE_SLEEP_TIME_MS:1000} # initial amount of time to wait between retries

2
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchConfig.java
View File

@ -26,7 +26,7 @@ import org.apache.skywalking.oap.server.library.module.ModuleConfig;
@Getter
@Setter
public class StorageModuleElasticsearchConfig extends ModuleConfig {
private String nameSpace;
private String namespace;
private String clusterNodes;
String protocol = "http";
/**

8
oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/StorageModuleElasticsearchProvider.java
View File

@ -110,10 +110,10 @@ public class StorageModuleElasticsearchProvider extends ModuleProvider {
public void prepare() throws ServiceNotProvidedException {
this.registerServiceImplementation(StorageBuilderFactory.class, new StorageBuilderFactory.Default());
if (StringUtil.isEmpty(config.getNameSpace())) {
config.setNameSpace("sw");
if (StringUtil.isEmpty(config.getNamespace())) {
config.setNamespace("sw");
} else {
config.setNameSpace(config.getNameSpace().toLowerCase());
config.setNamespace(config.getNamespace().toLowerCase());
}
if (config.getDayStep() > 1) {
TimeSeriesUtils.setDAY_STEP(config.getDayStep());
@ -155,7 +155,7 @@ public class StorageModuleElasticsearchProvider extends ModuleProvider {
elasticSearchClient = new ElasticSearchClient(
config.getClusterNodes(), config.getProtocol(), config.getTrustStorePath(), config
.getTrustStorePass(), config.getUser(), config.getPassword(),
indexNameConverter(config.getNameSpace()), config.getConnectTimeout(),
indexNameConverter(config.getNamespace()), config.getConnectTimeout(),
config.getSocketTimeout(), config.getNumHttpClientThread()
);
this.registerServiceImplementation(

2
tools/profile-exporter/application.yml
View File

@ -18,7 +18,7 @@ core:
storage:
selector: ${SW_STORAGE:h2}
elasticsearch:
nameSpace: ${SW_NAMESPACE:""}
namespace: ${SW_NAMESPACE:""}
clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
#trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}