* log: change log format to JSON payload for better log in Stackdriver
change the log format in Go written service from text payload to
JSON payload using 3rd party logging library (logrus).
https://cloud.google.com/logging/docs/structured-logging33a1e118e1/json_formatter.go (L40-L49)
Effected services are frontend, productcatalogservice, checkoutservice,
and shippinservice.
Also change target container registry and locust scenario for testing.
* revert kubernetes manifests to point to the original container registry URLs
* revert skaffold.yaml to point to the original registry
* loadgenerator: revert locust settings
* align all function names and messages to the official product name "Stackdriver"
Hipster Shop was demonstrated in front of rougly 700 individuals at Kubecon
EU. The demo itself was shown on 3 cloud providers (Azure, Google, On-prem).
The full speaking engagement recording:
https://www.youtube.com/watch?v=-t2BfT59zJA
* Add Stackdriver Profiler Python agent to EmailService and
RecommendationService
* Update recommendation_server.py
* Moved Profiler init to a function
* Moved Profiler init to a function
* Delete key.json
* Delete key.json
* Delete key.json
All mentioned projects (Kubernetes, Istio, etc.) point to their websites expect for Skaffold that points to its source code repository.
The website should be a better entry point to understand the big picture, access the quickstart guides and more in-depth howtos.
* remove per-yaml license headers in release manifests
* manually insert a license header in release manifests
* add README.md to /kubernetes-manifests indicating they're not ready to use
(and refer to /release instead).
* regenerate manifests
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
In certain situations (see details below), the deployment to Kubernetes fails with:
> "The Deployment [DEPLOYMENT_OBJECT] is invalid: [...] `selector` does not match template `labels`".
This is caused by the K8S Deployment manifests missing an explicit `selector` value.
This commit:
* adds explicit `selector` values for all Deployment objects.
* bumps the K8S API from the deprecated `extensions/v1beta1` version to the stable `apps/v1` version. This version made the `selector` property of the Deployment a required value, preventing any issues with missing selectors in the future.
This change is backwards compatible with existing deployments of the microservices demo app. I.e. you should be able to pull this change and run `skaffold run` against an existing deployment of the app without issues.
This will not however resolve the issue for existing deployments. Selectors are immutable and will therefore retain their current defaulted value. You should run `skaffold delete` followed by `skaffold run` after having pulled this change to do a clean re-deployment of the app, which will resolve the issue.
**The nitty-gritty details**
In the `extensions/v1beta1` version of K8S API (the version that was used by this project), the `selector` property of a Deployment object is optional and is defaulted to the labels used in the pod template. This can cause subtle issues leading to deployment failures. This project, where Deployment selectors were omitted, is a good example of what can go wrong with defaulted selectors.
Consider this:
1. Run `skaffold run` to build locally with Docker and deploy.
Since the Deployment specs don't have explict selectors, they will be defaulted to the pod template labels. And since skaffold adds additional labels to the pod template like `skaffold-builder` and `skaffold-deployer`, the end-result will be a selector that looks like this:
> app=cartservice,cleanup=true,docker-api-version=1.39,skaffold-builder=local,skaffold-deployer=kubectl,skaffold-tag-policy=git-commit,tail=true
So far, so good.
2. Now run `skaffold run -p gcb --default-repo=your-gcr-repo` to build on Google Cloud Build instead of building locally.
This will blow up when attempting to deploy to Kubernetes with an error similar to:
> The Deployment "cartservice" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"skaffold-builder":"google-cloud-build", "profiles"="gcb", "skaffold-deployer":"kubectl", "skaffold-tag-policy":"git-commit", "docker-api-version":"1.39", "tail":"true", "app":"cartservice", "cleanup":"true"}: `selector` does not match template `labels`
(and the same error for every other deployment object)
This is because the skaffold labels that were automatically added to the pod template have changed to include references to Google Cloud Build. That normally shouldn't be an issue.
But without explicit Deployment selectors, this results in the defaulted selectors for our Deployment objects to have also changed. Which means that the new version of our Deployment objects are now managing different sets of Pods. Which is thankfully caught by kubectl before the deployment happens (otherwise this would have resulted in orphaned pods).
In this commit, we explicitely set the `selector` value of all Deployment objects, which fixes this issue. We also bump the K8S API version to the stable `apps/v1`, which makes the `selector` property a required value and will avoid accidently forgetting selectors in the future.
More details if you're curious:
* Why defaulted Deployment selectors cause problems: https://github.com/kubernetes/kubernetes/issues/26202
* Why Deployment selectors should be (and were made) immutable: https://github.com/kubernetes/kubernetes/issues/50808
This commit sets the new sampling decision field that is recognized by the
Stackdriver Logging agent, "logging.googleapis.com/traceSampled". The sampling
decision field was added in
https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/pull/297, and
it won't be available until the new version of fluent-plugin-google-cloud is
used in GKE.
Previous release script wasn't committing/tagging in the right order.
Also made scripts resistant to:
* empty env vars
* current workdir the script is invoked from
cc: @m-okeefe
without &&, load generator startup was failing intermittently when it fails
to install curl from apk-add.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Closes#75.
### Changelog
Adds 4 scripts to the `hack/` directory for building/pushing images, injecting images tags into static manifests, and tagging new releases. See [hack/README.md](https://github.com/m-okeefe/microservices-demo/tree/release-script/hack).
**Note**: since we have not pushed images yet, the images in the `./release/` manifests are still set to the skaffold defaults (eg. `adservice`).
The Log4j JsonLayout puts the log entry timestamp in a field named "instant" by
default, but the Stackdriver Logging agent does not understand that field. The
logging agent instead uses the time that it received the log entry, which is
less accurate and has only second-level precision.
This commit adds a key-value pair to the JsonLayout pattern that can be
understood by the logging agent. It uses a "time" key as described in
https://cloud.google.com/logging/docs/agent/configuration#timestamp-processing
and formats the timestamp as described in the Protocol Buffer JSON mapping,
https://developers.google.com/protocol-buffers/docs/proto3#json.
Allowing the Stackdriver Logging agent to read the more accurate timestamps
inserted by Log4j is especially important in the adservice, because the logs are
correlated with traces, and it is important to see where each message was logged
on the timeline of the trace.
