Add docs of e2e development guide (#2984)

* Add docs of e2e development guide

docs/en/guides/README.md

@@ -26,6 +26,7 @@ All the following channels are open to the community, you could choose the way y
 ## For code developer
 For developers, first step, read [Compiling Guide](How-to-build.md). It teaches developer how to build the project in local and set up the environment.
 
+### Integration Tests
 After setting up the environment and writing your codes, in order to make it more easily accepted by SkyWalking project, you'll
 need to run the tests locally to verify that your codes don't break any existed features,
 and write some unit test (UT) codes to verify that the new codes work well, preventing them being broke by future contributors.
@@ -39,10 +40,52 @@ Therefore, to run the UTs, try `./mvnw clean test`, this will only run the UTs,
 If you want to run the ITs please activate the `CI-with-IT` profile
 as well as the the profiles of the modules whose ITs you want to run.
 e.g. if you want to run the ITs in `oap-server`, try `./mvnw -Pbackend,CI-with-IT clean verify`,
-and if you'd like to run all the ITs, simple run `./mvnw -Pall,CI-with-IT clean verify`.
+and if you'd like to run all the ITs, simply run `./mvnw -Pall,CI-with-IT clean verify`.
 
 Please be advised that if you're writing integration tests, name it with the pattern `IT*` to make them only run in `CI-with-IT` profile.
 
+### End to End Tests (E2E for short)
+Since version 6.3.0, we have introduced more automatic tests to perform software quality assurance, E2E is one of the most important parts.
+
+> End-to-end testing is a methodology used to test whether the flow of an application is performing as designed from start to finish.
+ The purpose of carrying out end-to-end tests is to identify system dependencies and to ensure that the right information is passed between various system components and systems.
+
+The e2e tests involves the OAP server, Web App, and the instrumented services, all of them run in a designed Docker container, connecting to each other; 
+in addition, there is a test controller running outside of the container that sends requests to the instrumented service,
+and then verifies the corresponding results after those requests, by GraphQL API of the SkyWalking Web App.
+
+Run the command `./mvnw -f test/e2e/pom.xml clean verify` under the root directory to get an intuition on how they work together(note that it may take a long time).
+
+#### Writing E2E Cases
+
+- Set up environment in Intellij IDEA
+
+The e2e test is an individual project under the SkyWalking root directory and the IDEA cannot recognize it by default, right click
+on the file `test/e2e/pom.xml` and click `Add as Maven Project`, things should be ready now.
+
+- Orchestrate the components
+
+Our goal of E2E tests is to test the SkyWalking project in a whole, including the OAP server, Web App, and even the frontend UI(not now),
+in single node mode as well as cluster mode, therefore the first step is to determine what case we are going to verify and orchestrate the 
+components.
+ 
+In order to make it more easily to orchestrate, we're using a [Docker e2e-container](https://github.com/SkyAPMTest/e2e-container) that can run
+OAP server in both standalone and cluster mode, it can also run the given instrumented services. Refer to the repository for more details;
+
+Basically you will need:
+1. (If in cluster mode) start up the third-party service containers that SkyWalking depends on, such as ZooKeeper as coordinator, ElasticSearch as storage, etc. in the docker-maven-plugin;
+1. (If in cluster mode) carefully map the addresses/ports into the e2e-container that can be used by the OAP server to connect to;
+1. Start the OAP server, Web App, and instrumented services in the e2e-container, mount your customized startup script to `/rc.d/` in e2e-container and it gets run when the container starts up;
+1. Use the utilities provided in e2e-container to check the healthiness of the components;
+1. Design the test controller and verify expected result;
+
+We've given a simple example that verifies SkyWalking should work as expected in cluster mode, refer to [the codes](../../../test/e2e/e2e-cluster) for detail.
+
+- Write test controller
+
+To put it simple, test controllers are basically tests that can be bound to the Maven `integration-test/verify` phase.
+They send **designed** requests to the instrumented service, and expect to get corresponding traces/metrics/metadata from the SkyWalking webapp GraphQL API.
+
 ### Project Extensions
 SkyWalking project supports many ways to extend existing features. If you are interesting in these ways,
 read the following guides.