Commit graph

351 commits

Author SHA1 Message Date
Kristen Kozak
ff7502fbab adservice: enable OpenCensus log correlation
OpenCensus log correlation
(https://github.com/census-instrumentation/opencensus-java/tree/master/contrib/log_correlation/stackdriver)
inserts the trace ID, span ID, and sampling decision into every log entry.  The
tracing data can be used to find all log entries associated with a single trace
or span.

This commit also upgrades OpenCensus to 0.16.1.
2018-09-26 20:24:29 -07:00
Kristen Kozak
338612caf7 adservice: export logs with the Stackdriver Logging library
This commit exports logs from the adservice directly to Stackdriver using the
Stackdriver Logging library.  Using the Stackdriver Logging library instead of
unstructured logging gives more control over setting fields such as severity.
This change can also be used to enable OpenCensus log correlation
(https://github.com/census-instrumentation/opencensus-java/tree/master/contrib/log_correlation/stackdriver).
2018-09-26 20:24:29 -07:00
sebright
c61a8aea4c adservice: upgrade grpc-java to 1.15.0 (#54)
Upgrading grpc-java fixed an error that I encountered when I tried modifying the adservice to write logs to Stackdriver with google-cloud-logging ("`com.google.cloud.logging.LoggingException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Credentials require channel with PRIVACY_AND_INTEGRITY security level. Observed security level: NONE`").
2018-09-26 19:33:52 -07:00
Ahmet Alp Balkan
3be5c82c73
skaffold: add gcb timeout (default is too low)
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-09-26 12:22:51 -07:00
Colin Nelson
7ced638e2f paymentservice: Docker image size optimization (#53)
Reduced docker image size to ~153MB (was ~781MB). 
* Uses alpine linux and multi stage builds. 
* Changed `CMD` to `ENTRYPOINT`
2018-09-26 10:58:33 -07:00
Colin Nelson
1263da2172 Reduced loadgenerator's image size (#52)
Reduce loadgenerator's image size from ~972MB to ~117MB

* Changed loadgen.sh to execute with `/bin/sh` as opposed to `/bin/bash`
* Changed dockerfile to a multi stage build
* Changed base image to `python:3-alpine` as opposed to `python:3.6`
2018-09-26 09:18:23 -07:00
Colin Nelson
2924250449 emailservice Image Optimization (#51)
Reduce docker image for emailservice to ~240 MB (down from ~ 1.31 GB)

Main application (`email_server.py`) now runs as python 2.7. Before we had both Python 2.7 and Python 3 installed in the image.

Switched to using `python:2.7-alpine3.8` as the base image, and used multi-stage dockerfiles to keep dependencies minimal.

Fixes #49 

From my shell:
```
$ docker build -t emailservice:dev . && docker run -it emailservice:dev
Sending build context to Docker daemon  97.28kB
Step 1/17 : FROM python:2.7-alpine3.8 as base
 ---> b2bc7255b42c
Step 2/17 : FROM base as builder
 ---> b2bc7255b42c
Step 3/17 : RUN apk add --update --no-cache     gcc     linux-headers     make     musl-dev     python-dev     g++     cairo-dev     cairo     openssl-dev     gobject-introspection-dev
 ---> Using cache
 ---> 6daf3d9fe49a
Step 4/17 : ENV GRPC_PYTHON_VERSION 1.15.0
 ---> Using cache
 ---> 3e33d97d9580
Step 5/17 : RUN python -m pip install --upgrade pip
 ---> Using cache
 ---> e8fa3879c282
Step 6/17 : RUN pip install grpcio==${GRPC_PYTHON_VERSION} grpcio-tools==${GRPC_PYTHON_VERSION}
 ---> Using cache
 ---> c6fba7743eed
Step 7/17 : COPY requirements.txt .
 ---> Using cache
 ---> 1f6b0a444980
Step 8/17 : RUN pip install -r requirements.txt
 ---> Using cache
 ---> 8cc0a7af6aa8
Step 9/17 : FROM base as final
 ---> b2bc7255b42c
Step 10/17 : RUN GRPC_HEALTH_PROBE_VERSION=v0.1.0-alpha.1 &&     wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 &&     chmod +x /bin/grpc_health_probe
 ---> Using cache
 ---> e954a0384081
Step 11/17 : ENV PYTHONUNBUFFERED=0
 ---> Using cache
 ---> 64ece3d72a66
Step 12/17 : WORKDIR /email_server
 ---> Using cache
 ---> 27b34dc14492
Step 13/17 : COPY --from=builder /usr/local/lib/python2.7/ /usr/local/lib/python2.7/
 ---> Using cache
 ---> 60035ec8dfd4
Step 14/17 : RUN apk add --no-cache libstdc++
 ---> Using cache
 ---> 920be90c126e
Step 15/17 : COPY . .
 ---> Using cache
 ---> 9541bed2d7a0
Step 16/17 : EXPOSE 8080
 ---> Using cache
 ---> 48fbeaa852b9
Step 17/17 : ENTRYPOINT [ "python", "email_server.py" ]
 ---> Using cache
 ---> ff317770992d
Successfully built ff317770992d
Successfully tagged emailservice:dev
starting the email service in dummy mode.
listening on port: 8080
```
2018-09-25 16:43:39 -07:00
Yoshi Yamaguchi
6460427bee Change log format to JSON and log destination to stdout (#48)
* log: change log format to JSON payload for better log in Stackdriver

change the log format in Go written service from text payload to
JSON payload using 3rd party logging library (logrus).

https://cloud.google.com/logging/docs/structured-logging
33a1e118e1/json_formatter.go (L40-L49)

Effected services are frontend, productcatalogservice, checkoutservice,
and shippinservice.

Also change target container registry and locust scenario for testing.

* revert kubernetes manifests to point to the original container registry URLs

* revert skaffold.yaml to point to the original registry

* loadgenerator: revert locust settings
2018-09-25 16:43:17 -07:00
Colin Nelson
ab601665d1 Reduced cartservice Docker Image size (#46)
Reduces docker image size for cartservice from ~2.36 GB to ~157 MB
Fixes Issue #36 

![image-size](https://user-images.githubusercontent.com/7820716/45908973-37a31e00-bdb4-11e8-92d3-d84acfb1c10f.png)
2018-09-24 09:54:25 -07:00
Ahmet Alp Balkan
3863f6d14d
recommendationsvc: optimize img (1.1 GB to 310 MB) (#45) 2018-09-21 15:40:25 -07:00
Simon Zeltser
1f60819dee Introducing super basic health check for cart service (#44)
* Introducing super basic health check for cart service
  - Generated C# proto implementation for grpc health check
  - Moved all C# protos to a dedicated folder
  - Implemented basic health checking to ping CartStore (which is Redis in default implementation)
  - Base plumbing for health checks

* Introducing super basic health check for cart service

- Generated C# proto implementation for grpc health check
- Moved all C# protos to a dedicated folder
- Implemented basic health checking to ping CartStore (which is Redis in default implementation)
- Base plumbing for health checks

* Changing Ping health probe to call Redis Cache Ping method
2018-09-21 12:09:52 -07:00
Ahmet Alp Balkan
1bab006af1
grpc: implement health check in go services (#27)
Also locked grpc-go to =1.14 exact version.
See grpc/grpc-go#2314 and grpc/grpc-go#2313 for reference
2018-09-20 10:31:31 -07:00
Ace Nassri
6c37a96f3a Add health checks to Node.js services (#26)
* Move Node healthchecks to gRPC

* gitignore proto files

* Switch to standard health RPC

* Fix lint

* Update client.js

* Add protos back + update them

* node services: fix & run genproto.sh

this gets currencyservice to work but paymentservice is still crashing
in the docker container.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>

* Fix docker breaking

* update dockerfiles with released health probe

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-09-19 12:35:22 -07:00
Ahmet Alp Balkan
360d983512
adservice: use grpc health checks (#29)
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-09-19 12:35:07 -07:00
Ahmet Alp Balkan
880ee16be2
grpc: add health checks to python services (#28)
also converted line endings for recommendationservice/requirements.txt from
dos to unix.
2018-09-19 12:34:56 -07:00
rghetia
fc6df2daea Remove monitored resource autodetect (#42) 2018-09-14 12:27:25 -07:00
Ahmet Alp Balkan
6d96944158
travis: add docker build for images (#38)
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-09-06 15:35:36 -07:00
Ahmet Alp Balkan
0f23d6bbf9
frontend: add simple /healthz endpoint
Decouple frontend health checks from GET /, which relies on other services to
be reachable. See #34.

Closes #34.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-09-04 22:44:05 -07:00
Ahmet Alp Balkan
efef44c2d7
README: fix syntax 2018-08-30 11:01:09 -07:00
Ahmet Alp Balkan
1ef223af17
README: fix syntax
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-30 10:59:57 -07:00
Ahmet Alp Balkan
ea02858fad
README: clarify gcr image updating
Looks like this is still confusing users.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-30 10:58:22 -07:00
Ahmet Alp Balkan
228105d229
frontend: add link to github repo from footer
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-30 10:37:58 -07:00
Ahmet Alp Balkan
7f5bb6c60b
frontend: add vendor/ to dockerignore
this is causing leftover local vendor/ to be used in docker builds

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-27 13:29:16 -07:00
Ahmet Alp Balkan
b9a978e84f
k8s/adservice: bump memory limits for adservice (#33)
adservice has beeing OOMkill'ed over the weekend. it looks like the memory
goes up to ~130MiB and then gets killed.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-27 11:30:47 -07:00
Eno Compton
342a2d5754 Resolve compilation error in frontend (#32)
Fixes #31
2018-08-27 09:47:13 -07:00
rghetia
7f2ecfea9e Instrument Go service with Stats and monitored resources. (#30)
added default views for stats  for http and grpc in Go services only.
added monitored resources for go.
2018-08-24 16:30:50 -07:00
Ahmet Alp Balkan
04c3876e94
frontend: add robots.txt to disallow
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-22 19:29:55 -07:00
Ahmet Alp Balkan
4683ee449c
README: add adservice to services list + diagram
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-22 17:11:53 -07:00
rghetia
f35fdbcac3 Initial commit for Ads Service. (#21)
* Initial commit for Ads Service.

* update comments for AdsService and AdsServiceClient

* Refactor Ads to Ad
Move building AdService to Docker
Use default setting for Stackdriver Exporter.
Add license text.

* Revert the projectId

- also remove commented code from frontend/rpc.go

* Add adservie to skaffold.yaml

* Remove skaffold-adservice.yaml

* Replace personal projectId with demo projectId.

* Fix the crash in adservice when ran in locally.

* Ignore .skaffold*yaml file and .kubernetes-manifests-*/ dir for easy ProjectID switch.

* Fixed review comments.
1. Changed Ad redirect urls to products.
2. Removed leftovers from Dockerfile/kub*manifests*yaml
3. Added retry for StackDriver.
4. Added log for Ad request.
5. Added comment for gradle caching.
6. Added README.md to src/adservice.

* Added GRPC Health service to Ad Service
Also added
1. timeout to getAd RPC call in frontend.
2. Async thread for stackdriver init.
2018-08-22 17:02:46 -07:00
Ahmet Alp Balkan
1d266bfdcf
Merge pull request #24 from ocervell/patch-1
Update skaffold.yaml to add gcb build
2018-08-21 10:29:32 -07:00
Ahmet Alp Balkan
074048ef5a
skaffold.yaml: annotate gcb profile 2018-08-21 10:27:35 -07:00
Olivier Cervello
0d6e54f812
Increaes GCB VM disk space to avoid build failure
Default VM disk size is 200 for Cloud Build, we need at least 300 to build all services.
2018-08-20 12:17:54 -05:00
Ahmet Alp Balkan
3c86418822
Merge pull request #25 from ahmetb/frontend-ads
frontend: add integration for adservice
2018-08-17 09:10:50 -07:00
Ahmet Alp Balkan
3e90b73464
frontend: add integration for adservice
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-16 18:01:52 -07:00
Ahmet Alp Balkan
c9cd627453
Merge pull request #23 from ahmetb/proto-fix
pb: fix style issue in Ads message
2018-08-16 13:44:37 -07:00
Olivier Cervello
a7a3bb7e03
Update skaffold.yaml
`skaffold run` will fail while building the `cartservice` with a 'No Space Left on Device Error'
Increasing the disk allocated for the build (default 200GB) to get rid of this error.
2018-08-16 15:26:34 -05:00
Ahmet Alp Balkan
390e489cd7
pb: fix style issue in Ads message
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-16 12:50:32 -07:00
Ahmet Alp Balkan
fad711b2db
update readme with detailed gke instructions
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-16 12:49:17 -07:00
Ahmet Alp Balkan
2cbe4b6f01
Merge pull request #22 from rghetia/ads_proto
Add proto for Ads Service.
2018-08-16 09:21:15 -07:00
rahulpa
24aaa3e376 Add proto for Ads Service. 2018-08-16 09:11:26 -07:00
Ahmet Alp Balkan
4e9d46c8e7
img: make the screenshot sizes equal
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-14 12:00:20 -07:00
Ahmet Alp Balkan
00fc397eca
README: add screenshots, diagrams, features list
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-13 11:28:15 -07:00
Ahmet Alp Balkan
98885b8776
Update README.md 2018-08-09 14:40:43 -07:00
Ahmet Alp Balkan
e5a04449a0
Merge pull request #15 from dgageot/d4d-stable-supports-k8s
Docker for Desktop stable supports k8s now
2018-08-06 16:16:42 -07:00
David Gageot
5bdfe5dfc7 Docker for Desktop stable supports k8s now
Signed-off-by: David Gageot <david@gageot.net>
2018-08-05 09:35:35 +02:00
Ahmet Alp Balkan
db40889ad0
Change gcr projectID in images
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-04 11:23:35 -07:00
Ahmet Alp Balkan
427f13fcd3
go: use dep in builds, remove relative imports
Fixes #11.
Fixes #12.
Fixes #13.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-04 11:21:03 -07:00
Ahmet Alp Balkan
c7dd0cbf31
test-cli: remove
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-08-04 11:12:21 -07:00
Ahmet Alp Balkan
4572974c88
use dep to lock Go dependencies (without vendor/)
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-07-30 19:18:05 -07:00
Ahmet Alp Balkan
58e1879e33
rm .vscode/ directory
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2018-07-30 19:11:53 -07:00