Active Monitoring (GCloud)
To be proactive in detecting any issues before or when they occur, active monitoring needs to be in place. The areas where we recommend to monitor are:
- (Minimum Required) ETH balance of the wallet address assigned to the node.
- Operator UI port to be open and responsive. (Usually: 6688)
- Ethereum http and websocket ports to be open and responsive. (Usually: 8545 & 8546)
- Ethereum client disk, RAM and CPU usage.
Monitoring can be set up from the Docker container’s output and fed into most major logging providers. For example, you can use Docker’s docs to set up the logging driver for GCloud (*See steps below)
JSON_CONSOLE configuration variable – Change to TRUE
Flag to print the console output in JSON or in human-friendly output.
*Google Cloud Logging driver
The Google Cloud Logging driver sends container logs to Google Cloud Logging Logging.
To use the gcplogs driver as the default logging driver, set the log-driver and log-opt keys to appropriate values in the daemon.json file, which is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\daemon.json on Windows Server. For more about configuring Docker using daemon.json, see daemon.json.
The following example sets the log driver to gcplogs and sets the gcp-meta-name option.
Restart Docker for the changes to take effect.
You can set the logging driver for a specific container by using the –log-driver option to docker run:
docker run –log-driver=gcplogs …
This log driver does not implement a reader so it is incompatible with docker logs.
If Docker detects that it is running in a Google Cloud Project, it discovers configuration from the instance metadata service. Otherwise, the user must specify which project to log to using the –gcp-project log option and Docker attempts to obtain credentials from the Google Application Default Credential. The –gcp-project flag takes precedence over information discovered from the metadata server so a Docker daemon running in a Google Cloud Project can be overridden to log to a different Google Cloud Project using –gcp-project.
Docker fetches the values for zone, instance name and instance ID from Google Cloud metadata server. Those values can be provided via options if metadata server is not available. They do not override the values from metadata server.
You can use the –log-opt NAME=VALUE flag to specify these additional Google Cloud Logging driver options:
|gcp-project||optional||Which GCP project to log to. Defaults to discovering this value from the GCE metadata service.|
|gcp-log-cmd||optional||Whether to log the command that the container was started with. Defaults to false.|
|labels||optional||Comma-separated list of keys of labels, which should be included in message, if these labels are specified for the container.|
|env||optional||Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for the container.|
|env-regex||optional||Similar to and compatible with env. A regular expression to match logging-related environment variables. Used for advanced log tag options.|
|gcp-meta-zone||optional||Zone name for the instance.|
If there is collision between label and env keys, the value of the env takes precedence. Both options add additional fields to the attributes of a logging message.
Below is an example of the logging options required to log to the default logging destination which is discovered by querying the GCE metadata server.
docker run –log-driver=gcplogs \
–log-opt labels=location \
–log-opt env=TEST \
–log-opt gcp-log-cmd=true \
–env “TEST=false” \
–label location=west \
This configuration also directs the driver to include in the payload the label location, the environment variable ENV, and the command used to start the container.
An example of the logging options for running outside of GCE (the daemon must be configured with GOOGLE_APPLICATION_CREDENTIALS):
docker run –log-driver=gcplogs \
–log-opt gcp-meta-zone=west1 \
–log-opt gcp-meta-name=`hostname` \