change the log format in Python and Node.js services.
Effected services are currencyservice, emailservice, paymentservice,
and recommendationservice. Loadgenerator is left as is because of
the diffculty to change the log format and log target in locust.
ref. #47
The ad service now returns ads matching the categories of the product that is
currently displayed. Changes in this commit:
- List all products' categories in products.json.
- Pass the current product's categories from the frontend to the ad service when
looking up ads.
- Store a statically initialized multimap from product category to ad in the ad
service.
- Return all ads matching the given categories when handling an ads request.
The ad service continues to return random ads when no categories are given or
no ads match the categories.
This field can be used as the context keys to look up relevant ads in the ad
service.
/cc @rghetia
I also ran the genproto.sh scripts for the Java and Go services and included those changes in the second commit. I encountered an issue when I ran genproto.sh for the recommendation service, and I'm still looking into it.
Upgrading grpc-java fixed an error that I encountered when I tried modifying the adservice to write logs to Stackdriver with google-cloud-logging ("`com.google.cloud.logging.LoggingException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Credentials require channel with PRIVACY_AND_INTEGRITY security level. Observed security level: NONE`").
Reduce loadgenerator's image size from ~972MB to ~117MB
* Changed loadgen.sh to execute with `/bin/sh` as opposed to `/bin/bash`
* Changed dockerfile to a multi stage build
* Changed base image to `python:3-alpine` as opposed to `python:3.6`
Reduce docker image for emailservice to ~240 MB (down from ~ 1.31 GB)
Main application (`email_server.py`) now runs as python 2.7. Before we had both Python 2.7 and Python 3 installed in the image.
Switched to using `python:2.7-alpine3.8` as the base image, and used multi-stage dockerfiles to keep dependencies minimal.
Fixes#49
From my shell:
```
$ docker build -t emailservice:dev . && docker run -it emailservice:dev
Sending build context to Docker daemon 97.28kB
Step 1/17 : FROM python:2.7-alpine3.8 as base
---> b2bc7255b42c
Step 2/17 : FROM base as builder
---> b2bc7255b42c
Step 3/17 : RUN apk add --update --no-cache gcc linux-headers make musl-dev python-dev g++ cairo-dev cairo openssl-dev gobject-introspection-dev
---> Using cache
---> 6daf3d9fe49a
Step 4/17 : ENV GRPC_PYTHON_VERSION 1.15.0
---> Using cache
---> 3e33d97d9580
Step 5/17 : RUN python -m pip install --upgrade pip
---> Using cache
---> e8fa3879c282
Step 6/17 : RUN pip install grpcio==${GRPC_PYTHON_VERSION} grpcio-tools==${GRPC_PYTHON_VERSION}
---> Using cache
---> c6fba7743eed
Step 7/17 : COPY requirements.txt .
---> Using cache
---> 1f6b0a444980
Step 8/17 : RUN pip install -r requirements.txt
---> Using cache
---> 8cc0a7af6aa8
Step 9/17 : FROM base as final
---> b2bc7255b42c
Step 10/17 : RUN GRPC_HEALTH_PROBE_VERSION=v0.1.0-alpha.1 && wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && chmod +x /bin/grpc_health_probe
---> Using cache
---> e954a0384081
Step 11/17 : ENV PYTHONUNBUFFERED=0
---> Using cache
---> 64ece3d72a66
Step 12/17 : WORKDIR /email_server
---> Using cache
---> 27b34dc14492
Step 13/17 : COPY --from=builder /usr/local/lib/python2.7/ /usr/local/lib/python2.7/
---> Using cache
---> 60035ec8dfd4
Step 14/17 : RUN apk add --no-cache libstdc++
---> Using cache
---> 920be90c126e
Step 15/17 : COPY . .
---> Using cache
---> 9541bed2d7a0
Step 16/17 : EXPOSE 8080
---> Using cache
---> 48fbeaa852b9
Step 17/17 : ENTRYPOINT [ "python", "email_server.py" ]
---> Using cache
---> ff317770992d
Successfully built ff317770992d
Successfully tagged emailservice:dev
starting the email service in dummy mode.
listening on port: 8080
```
* log: change log format to JSON payload for better log in Stackdriver
change the log format in Go written service from text payload to
JSON payload using 3rd party logging library (logrus).
https://cloud.google.com/logging/docs/structured-logging33a1e118e1/json_formatter.go (L40-L49)
Effected services are frontend, productcatalogservice, checkoutservice,
and shippinservice.
Also change target container registry and locust scenario for testing.
* revert kubernetes manifests to point to the original container registry URLs
* revert skaffold.yaml to point to the original registry
* loadgenerator: revert locust settings
* Introducing super basic health check for cart service
- Generated C# proto implementation for grpc health check
- Moved all C# protos to a dedicated folder
- Implemented basic health checking to ping CartStore (which is Redis in default implementation)
- Base plumbing for health checks
* Introducing super basic health check for cart service
- Generated C# proto implementation for grpc health check
- Moved all C# protos to a dedicated folder
- Implemented basic health checking to ping CartStore (which is Redis in default implementation)
- Base plumbing for health checks
* Changing Ping health probe to call Redis Cache Ping method
* Move Node healthchecks to gRPC
* gitignore proto files
* Switch to standard health RPC
* Fix lint
* Update client.js
* Add protos back + update them
* node services: fix & run genproto.sh
this gets currencyservice to work but paymentservice is still crashing
in the docker container.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
* Fix docker breaking
* update dockerfiles with released health probe
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Decouple frontend health checks from GET /, which relies on other services to
be reachable. See #34.
Closes#34.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
adservice has beeing OOMkill'ed over the weekend. it looks like the memory
goes up to ~130MiB and then gets killed.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
* Initial commit for Ads Service.
* update comments for AdsService and AdsServiceClient
* Refactor Ads to Ad
Move building AdService to Docker
Use default setting for Stackdriver Exporter.
Add license text.
* Revert the projectId
- also remove commented code from frontend/rpc.go
* Add adservie to skaffold.yaml
* Remove skaffold-adservice.yaml
* Replace personal projectId with demo projectId.
* Fix the crash in adservice when ran in locally.
* Ignore .skaffold*yaml file and .kubernetes-manifests-*/ dir for easy ProjectID switch.
* Fixed review comments.
1. Changed Ad redirect urls to products.
2. Removed leftovers from Dockerfile/kub*manifests*yaml
3. Added retry for StackDriver.
4. Added log for Ad request.
5. Added comment for gradle caching.
6. Added README.md to src/adservice.
* Added GRPC Health service to Ad Service
Also added
1. timeout to getAd RPC call in frontend.
2. Async thread for stackdriver init.
`skaffold run` will fail while building the `cartservice` with a 'No Space Left on Device Error'
Increasing the disk allocated for the build (default 200GB) to get rid of this error.
