Basic Syntax
Each query is composed of one or more clauses, which can be combined using boolean operators. The basic structure of a query is as follows:field: The log field to query (e.g.,time,resource.labels.container_group_name,json_log.custom_field).operator: The comparison operator to use (e.g.,=,!=,>,<,contains).value: The value to compare against (e.g., a string, number, or date).
Log Schema
Log events are JSON objects with the following structure:- Note that a log event will have EITHER a
text_logOR ajson_log, but not both. If your application emits a log that is valid JSON, it will be parsed and made available viajson_log. Otherwise, the raw text will available intext_log. - No operators are allowed to query on
json_logdirectly, however you can query on any specific json fields withinjson_log, e.g.json_log.my_field. - You can do full-text search independent of structured fields by querying a special
logfield. Thelogfield only supports thecontains,!contains,contains~, and!contains~operators. It searches across bothtext_logandjson_logfields. - The
timefield indicates the system time on the node where the log was emitted. - The
receive_timefield indicates the time when the log was received. - Any of these fields can be used in the query language.
- Time and Date values take the form
YYYY-MM-DDTHH:MM:SSZ, and are expressed in UTC.
Resource Types And Labels
You can query logs from three different resource types, indicated in the fieldresource.type:
container: Logs from a running container instance. This includes all logs emitted to stdout and stderr within the container.instance_controller: System events that occur on the instance level, such as container start and stop events, probes passing and failing, etc.deployment_controller: System events that occur on the container group level, such as adjusting replica count, and starting and stopping container groups.
project_name: The name of your project in SaladCloud.container_group_name: The name of your container group in SaladCloud.
container and instance_controller types have the following additional labels:
container_group_version: The version of your container group when the log was emitted.instance_id: A unique ID for a specific instantiation of a container instance. This is independent frommachine_id, because a single machine ID can have multiple instance IDs associated with it, if a container has recreated or restarted a number of times on the same machine. In such a situation, each “run” has its owninstance_id.machine_id: The unique ID of the node running the container that created the log message.
Operators
The following operators are supported in log queries:=: Equality (Case Sensitive)=~: Equality (Case Insensitive)!=: Inequality (Case Sensitive)!=~: Inequality (Case Insensitive)>: Greater than.<: Less than.>=: Greater than or equal to.<=: Less than or equal to.contains: String contains (Case Sensitive)contains~: String contains (Case Insensitive)!contains: String does not contain (Case Sensitive)!contains~: String does not contain (Case Insensitive)startswith: String starts-with (Case Sensitive)startswith~: String starts-with (Case Insensitive)!startswith: String does not start with (Case Sensitive)!startswith~: String does not start with (Case Insensitive)endswith: String ends-with (Case Sensitive)endswith~: String ends-with (Case Insensitive)!endswith: String does not end with (Case Sensitive)!endswith~: String does not end with (Case Insensitive)
Combining Statements
Query statements can be combined with the following boolean operators:and: Logical ANDor: Logical ORnot: Logical NOT
Field Promotion in Structured Logs
If you emit structured logs (meaning logs that are valid JSON) with any of the following fields, it will be removed from thejson_log field and promoted to a top-level field in the log entry.
severityseverity_numbertrace_idspan_idparent_span_id
time or receive_timefield in your structured log WILL NOT override the respective time
fields of the log entry.
Troubleshooting Common Issues
- Ensure that quotes are properly escaped in your
.queryfield. - Ensure that times are specified in UTC and follow the ISO 8601 format, i.e.
YYYY-MM-DDTHH:MM:SSZ - Ensure that times are within the retention period of your logs.
- Ensure that times are in the past, i.e. no future times.