kazusa
/
skywalking-java
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add logging module to load log configuration dynamically (#7377) 

* Add logging module to load log configuration dynamically

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
This commit is contained in:
Gao Hongtao 2021-07-29 19:55:22 +08:00 committed by GitHub
parent 65e8ab3ecc
commit 1f28d2036a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 260 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
CHANGES.md
View File

@ -122,7 +122,9 @@ Release Notes.
* Filtering NaN value samples when build SampleFamily
* Add Thread and ClassLoader Metrics for the self-observability and otel-oc-rules
* Simple optimization of trace sql query statement. Avoid "select *" query method
* Breaking Change: rename `qps` to `rpm` in LAL
* Introduce dynamical logging to update log configuration at runtime
* Fix Kubernetes ConfigMap configuration center doesn't send delete event
* Breaking Change: emove `qps` and add `rpm` in LAL
#### UI

1
docs/en/setup/backend/dynamic-config.md
View File

@ -11,6 +11,7 @@ Right now, SkyWalking supports following dynamic configurations.
|alarm.default.alarm-settings| The alarm settings, will override `alarm-settings.yml`. | same as [`alarm-settings.yml`](backend-alarm.md) |
|core.default.apdexThreshold| The apdex threshold settings, will override `service-apdex-threshold.yml`. | same as [`service-apdex-threshold.yml`](apdex-threshold.md) |
|core.default.endpoint-name-grouping| The endpoint name grouping setting, will override `endpoint-name-grouping.yml`. | same as [`endpoint-name-grouping.yml`](endpoint-grouping-rules.md) |
|core.default.log4j-xml| The log4j xml configuration, will override `log4j2.xml`. | same as [`log4j2.xml`](dynamical-logging.md) |
|agent-analyzer.default.sampleRate| Trace sampling , override `receiver-trace/default/sampleRate` of `application.yml`. | 10000 |
|agent-analyzer.default.slowTraceSegmentThreshold| Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond. override `receiver-trace/default/slowTraceSegmentThreshold` of `application.yml`. | -1 |
|configuration-discovery.default.agentConfigurations| The ConfigurationDiscovery settings | look at [`configuration-discovery.md`](../service-agent/java-agent/configuration-discovery.md) |

52
docs/en/setup/backend/dynamical-logging.md Normal file
View File

@ -0,0 +1,52 @@
# Dynamical Logging
OAP server leverage the log4j2 to manage the logging system. The log4j2 supports changing the configuration
at the runtime, but you have to update the XML configuration file manually, which could waste you much time and
easy to make mistakes.
The dynamical logging, which depends on the dynamic configuration, provides us an agile way to update all OAP log4j
configurations through a single operation.
The key of the configuration item is `core.default.log4j-xml`, and you can select any one of the configuration implements
to store the content of log4j.xml. In the booting phase, once the core module gets started, the `core.default.log4j-xml`
would come into the OAP log4j context.
Supposing changing the configuration after OAP started, you have to wait a while to get the changes applied.
The default value is `60` seconds which you could change through `configuration.<configuration implement>.peroid` in application.yaml.
If you remove `core.default.log4j-xml` from the configuration center or disable the configuration module, the `log4j.xml`
laid in `config` directory would get the effect.
> Caveat: OAP only supports XML configuration format.
There is an example to show how to config dynamical logging through a ConfigMap in a Kubernetes cluster. Other configuration
clusters could follow the same convention to set up.
```yaml
apiVersion: v1
data:
core.default.log4j-xml: |-
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout charset="UTF-8" pattern="%d - %c - %L [%t] %-5p %x - %m%n"/>
</Console>
</Appenders>
<Loggers>
<logger name="io.grpc.netty" level="INFO"/>
<logger name="org.apache.skywalking.oap.server.configuration.api" level="TRACE"/>
<logger name="org.apache.skywalking.oap.server.configuration.configmap" level="DEBUG"/>
<Root level="WARN">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
kind: ConfigMap
metadata:
labels:
app: collector
release: skywalking
name: skywalking-oap
namespace: default
```

2
docs/menu.yml
View File

@ -149,6 +149,8 @@ catalog:
path: "/en/setup/backend/advanced-deployment"
- name: "Data Lifecycle. Time To Live (TTL)"
path: "/en/setup/backend/ttl"
- name: "Dynamical Logging"
path: "/en/setup/backend/dynamical-logging"
- name: "Deploy In Kubernetes"
path: "/en/setup/backend/backend-k8s"
- name: "Tracing"

5
oap-server/server-configuration/configuration-k8s-configmap/src/main/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigurationWatcherRegister.java
View File

@ -19,7 +19,6 @@
package org.apache.skywalking.oap.server.configuration.configmap;
import io.kubernetes.client.openapi.models.V1ConfigMap;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import lombok.extern.slf4j.Slf4j;
@ -46,9 +45,7 @@ public class ConfigmapConfigurationWatcherRegister extends ConfigWatcherRegister
if (log.isDebugEnabled()) {
log.debug("read config: name:{} ,value:{}", name, value);
}
if (Objects.nonNull(value)) {
configTable.add(new ConfigTable.ConfigItem(name, value));
}
configTable.add(new ConfigTable.ConfigItem(name, value));
}
return Optional.of(configTable);
}

8
oap-server/server-configuration/configuration-k8s-configmap/src/test/java/org/apache/skywalking/oap/server/configuration/configmap/ConfigmapConfigWatcherRegisterTest.java
View File

@ -67,7 +67,9 @@ public class ConfigmapConfigWatcherRegisterTest {
Assert.assertTrue(optionalConfigTable.isPresent());
ConfigTable configTable = optionalConfigTable.get();
Assert.assertEquals(configTable.getItems().size(), 0);
Assert.assertEquals(configTable.getItems().size(), 1);
Assert.assertEquals(configTable.getItems().get(0).getName(), "key1");
Assert.assertNull(configTable.getItems().get(0).getValue());
}
@Test
@ -79,7 +81,9 @@ public class ConfigmapConfigWatcherRegisterTest {
Assert.assertTrue(optionalConfigTable.isPresent());
ConfigTable configTable = optionalConfigTable.get();
Assert.assertEquals(configTable.getItems().size(), 0);
Assert.assertEquals(configTable.getItems().size(), 1);
Assert.assertEquals(configTable.getItems().get(0).getName(), "key1");
Assert.assertNull(configTable.getItems().get(0).getValue());
}
@Test

4
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/CoreModuleProvider.java
View File

@ -49,6 +49,7 @@ import org.apache.skywalking.oap.server.core.config.NamingControl;
import org.apache.skywalking.oap.server.core.config.group.EndpointNameGrouping;
import org.apache.skywalking.oap.server.core.config.group.EndpointNameGroupingRuleWatcher;
import org.apache.skywalking.oap.server.core.config.group.openapi.EndpointGroupingRuleReader4Openapi;
import org.apache.skywalking.oap.server.core.logging.LoggingConfigWatcher;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplateInitializer;
import org.apache.skywalking.oap.server.core.management.ui.template.UITemplateManagementService;
import org.apache.skywalking.oap.server.core.oal.rt.DisableOALDefine;
@ -121,6 +122,7 @@ public class CoreModuleProvider extends ModuleProvider {
private ApdexThresholdConfig apdexThresholdConfig;
private EndpointNameGroupingRuleWatcher endpointNameGroupingRuleWatcher;
private OALEngineLoaderService oalEngineLoaderService;
private LoggingConfigWatcher loggingConfigWatcher;
public CoreModuleProvider() {
super();
@ -307,6 +309,7 @@ public class CoreModuleProvider extends ModuleProvider {
TopNStreamProcessor.getInstance().setTopNWorkerReportCycle(moduleConfig.getTopNReportPeriod());
apdexThresholdConfig = new ApdexThresholdConfig(this);
ApdexMetrics.setDICT(apdexThresholdConfig);
loggingConfigWatcher = new LoggingConfigWatcher(this);
}
@Override
@ -349,6 +352,7 @@ public class CoreModuleProvider extends ModuleProvider {
DynamicConfigurationService.class);
dynamicConfigurationService.registerConfigChangeWatcher(apdexThresholdConfig);
dynamicConfigurationService.registerConfigChangeWatcher(endpointNameGroupingRuleWatcher);
dynamicConfigurationService.registerConfigChangeWatcher(loggingConfigWatcher);
}
@Override

99
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/LoggingConfigWatcher.java Normal file
View File

@ -0,0 +1,99 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.logging;
import com.google.common.base.Strings;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import lombok.extern.slf4j.Slf4j;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.config.ConfigurationSource;
import org.apache.skywalking.oap.server.configuration.api.ConfigChangeWatcher;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.logging.log4j.OapConfiguration;
import org.apache.skywalking.oap.server.library.module.ModuleProvider;
/**
* LoggingConfigWatcher watches the change of logging configuration. Once got the change content, it would apply them to
* the current logger context.
*/
@Slf4j
public class LoggingConfigWatcher extends ConfigChangeWatcher {
private final LoggerContext ctx;
private final OapConfiguration originConfiguration;
private volatile String content;
public LoggingConfigWatcher(final ModuleProvider provider) {
super(CoreModule.NAME, provider, "log4j-xml");
this.ctx = (LoggerContext) LogManager.getContext(false);
this.originConfiguration = (OapConfiguration) ctx.getConfiguration();
}
@Override
public void notify(final ConfigChangeEvent value) {
String newValue;
if (EventType.DELETE.equals(value.getEventType())) {
this.content = "";
newValue = "";
} else {
newValue = value.getNewValue();
}
try {
if (!reconfigure(newValue)) {
return;
}
} catch (Throwable t) {
log.error("failed to apply configuration to log4j", t);
return;
}
StringBuilder builder = new StringBuilder();
ctx.getConfiguration().getLoggers().forEach((loggerName, config) -> {
builder.append(Strings.isNullOrEmpty(loggerName) ? "Root" : loggerName)
.append(":")
.append(config.getLevel())
.append(",");
});
this.content = builder.toString();
}
@Override
public String value() {
return this.content;
}
private boolean reconfigure(final String newValue) {
if (Strings.isNullOrEmpty(newValue)) {
if (ctx.getConfiguration().equals(originConfiguration)) {
return false;
}
ctx.onChange(originConfiguration);
return true;
}
OapConfiguration oc;
try {
oc = new OapConfiguration(ctx, new ConfigurationSource(new ByteArrayInputStream(newValue.getBytes())));
} catch (IOException e) {
throw new RuntimeException(String.format("failed to parse %s from configuration center", newValue), e);
}
oc.initialize();
ctx.onChange(oc);
return true;
}
}

33
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfiguration.java Normal file
View File

@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.logging.log4j;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.config.ConfigurationSource;
import org.apache.logging.log4j.core.config.xml.XmlConfiguration;
/**
* OapConfiguration is the object loaded into log4j context.
*/
public class OapConfiguration extends XmlConfiguration {
public OapConfiguration(final LoggerContext loggerContext,
final ConfigurationSource configSource) {
super(loggerContext, configSource);
}
}

59
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/logging/log4j/OapConfigurationFactory.java Normal file
View File

@ -0,0 +1,59 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.logging.log4j;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.config.Configuration;
import org.apache.logging.log4j.core.config.ConfigurationFactory;
import org.apache.logging.log4j.core.config.ConfigurationSource;
import org.apache.logging.log4j.core.config.Order;
import org.apache.logging.log4j.core.config.plugins.Plugin;
/**
* OapConfigurationFactory guarantee the log4j configuration object is {@link OapConfiguration}.
* It guarantees the downcast {@link Configuration} to {@link OapConfiguration} safely.
*/
@Plugin(name = "OapConfigurationFactory", category = "ConfigurationFactory")
@Order(10)
public class OapConfigurationFactory extends ConfigurationFactory {
/**
* Valid file extensions for XML files.
*/
public static final String[] SUFFIXES = new String[] {".xml", "*"};
/**
* Returns the file suffixes for XML files.
* @return An array of File extensions.
*/
@Override
public String[] getSupportedTypes() {
return SUFFIXES;
}
/**
* Return the configuration.
* @param loggerContext The logger context
* @param source The input source
* @return the configuration
*/
@Override
public Configuration getConfiguration(final LoggerContext loggerContext, final ConfigurationSource source) {
return new OapConfiguration(loggerContext, source);
}
}