**Online Boutique** is a cloud-native microservices demo application. Online Boutique consists of a 10-tier microservices application. The application is a web-based e-commerce app where users can browse items, add them to the cart, and purchase them. **Google uses this application to demonstrate use of technologies like Kubernetes/GKE, Istio, Stackdriver, gRPC and OpenCensus**. This application works on any Kubernetes cluster (such as a local one), as well as Google Kubernetes Engine. It’s **easy to deploy with little to no configuration**. If you’re using this demo, please **★Star** this repository to show your interest! > 👓**Note to Googlers:** Please fill out the form at > [go/microservices-demo](http://go/microservices-demo) if you are using this > application. Looking for the old Hipster Shop frontend interface? Use the [manifests](https://github.com/GoogleCloudPlatform/microservices-demo/tree/v0.1.4/kubernetes-manifests) in release [v0.1.4](https://github.com/GoogleCloudPlatform/microservices-demo/releases/v0.1.4). ## Screenshots | Home Page | Checkout Screen | | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | [](./docs/img/online-boutique-frontend-1.png) | [](./docs/img/online-boutique-frontend-2.png) | ## Service Architecture **Online Boutique** is composed of many microservices written in different languages that talk to each other over gRPC. [](./docs/img/architecture-diagram.png) Find **Protocol Buffers Descriptions** at the [`./pb` directory](./pb). | Service | Language | Description | | ---------------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------- | | [frontend](./src/frontend) | Go | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. | | [cartservice](./src/cartservice) | C# | Stores the items in the user's shopping cart in Redis and retrieves it. | | [productcatalogservice](./src/productcatalogservice) | Go | Provides the list of products from a JSON file and ability to search products and get individual products. | | [currencyservice](./src/currencyservice) | Node.js | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. | | [paymentservice](./src/paymentservice) | Node.js | Charges the given credit card info (mock) with the given amount and returns a transaction ID. | | [shippingservice](./src/shippingservice) | Go | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock) | | [emailservice](./src/emailservice) | Python | Sends users an order confirmation email (mock). | | [checkoutservice](./src/checkoutservice) | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. | | [recommendationservice](./src/recommendationservice) | Python | Recommends other products based on what's given in the cart. | | [adservice](./src/adservice) | Java | Provides text ads based on given context words. | | [loadgenerator](./src/loadgenerator) | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. | ## Features - **[Kubernetes](https://kubernetes.io)/[GKE](https://cloud.google.com/kubernetes-engine/):** The app is designed to run on Kubernetes (both locally on "Docker for Desktop", as well as on the cloud with GKE). - **[gRPC](https://grpc.io):** Microservices use a high volume of gRPC calls to communicate to each other. - **[Istio](https://istio.io):** Application works on Istio service mesh. - **[OpenCensus](https://opencensus.io/) Tracing:** Most services are instrumented using OpenCensus trace interceptors for gRPC/HTTP. - **[Stackdriver APM](https://cloud.google.com/stackdriver/):** Many services are instrumented with **Profiling**, **Tracing** and **Debugging**. In addition to these, using Istio enables features like Request/Response **Metrics** and **Context Graph** out of the box. When it is running out of Google Cloud, this code path remains inactive. - **[Skaffold](https://skaffold.dev):** Application is deployed to Kubernetes with a single command using Skaffold. - **Synthetic Load Generation:** The application demo comes with a background job that creates realistic usage patterns on the website using [Locust](https://locust.io/) load generator. ## Installation We offer the following installation methods: 1. **Running locally** (~20 minutes) You will build and deploy microservices images to a single-node Kubernetes cluster running on your development machine. There are three options to run a Kubernetes cluster locally for this demo: - [Minikube](https://github.com/kubernetes/minikube). Recommended for Linux hosts (also supports Mac/Windows). - [Docker for Desktop](https://www.docker.com/products/docker-desktop). Recommended for Mac/Windows. - [Kind](https://www.docker.com/products/docker-desktop). Supports Mac/Windows/Linux. 1. **Running on Google Kubernetes Engine (GKE)”** (~30 minutes) You will build, upload and deploy the container images to a Kubernetes cluster on Google Cloud. 1. **Using pre-built container images:** (~10 minutes, you will still need to follow one of the steps above up until `skaffold run` command). With this option, you will use pre-built container images that are available publicly, instead of building them yourself, which takes a long time). ### Option 1: Running locally > 💡 Recommended if you're planning to develop the application or giving it a > try on your local cluster. 1. Install tools to run a Kubernetes cluster locally: - kubectl (can be installed via `gcloud components install kubectl`) - Local Kubernetes cluster deployment tool: - [Minikube (recommended for Linux)](https://kubernetes.io/docs/setup/minikube/) - [Docker for Desktop (recommended for Mac/Windows)](https://www.docker.com/products/docker-desktop) - It provides Kubernetes support as [noted here](https://docs.docker.com/docker-for-mac/kubernetes/) - [Kind](https://github.com/kubernetes-sigs/kind) - [skaffold]( https://skaffold.dev/docs/install/) (ensure version ≥v0.20) 1. Launch the local Kubernetes cluster with one of the following tools: - To launch **Minikube** (tested with Ubuntu Linux). Please, ensure that the local Kubernetes cluster has at least: - 4 CPU's - 4.0 GiB memory ```shell minikube start --cpus=4 --memory 4096 ``` - To launch **Docker for Desktop** (tested with Mac/Windows). Go to Preferences: - choose “Enable Kubernetes”, - set CPUs to at least 3, and Memory to at least 6.0 GiB - on the "Disk" tab, set at least 32 GB disk space - To launch a **Kind** cluster: ```shell kind create cluster ``` 1. Run `kubectl get nodes` to verify you're connected to “Kubernetes on Docker”. 1. Run `skaffold run` (first time will be slow, it can take ~20 minutes). This will build and deploy the application. If you need to rebuild the images automatically as you refactor the code, run `skaffold dev` command. 1. Run `kubectl get pods` to verify the Pods are ready and running. 1. Access the web frontend through your browser - **Minikube** requires you to run a command to access the frontend service: ```shell minikube service frontend-external ``` - **Docker For Desktop** should automatically provide the frontend at http://localhost:80 - **Kind** does not provision an IP address for the service. You must run a port-forwarding process to access the frontend at http://localhost:8080: ```shell kubectl port-forward deployment/frontend 8080:8080 ``` ### Option 2: Running on Google Kubernetes Engine (GKE) > 💡 Recommended if you're using Google Cloud Platform and want to try it on > a realistic cluster. 1. Install tools specified in the previous section (Docker, kubectl, skaffold) 1. Create a Google Kubernetes Engine cluster and make sure `kubectl` is pointing to the cluster. ```sh gcloud services enable container.googleapis.com ``` ```sh gcloud container clusters create demo --enable-autoupgrade \ --enable-autoscaling --min-nodes=3 --max-nodes=10 --num-nodes=5 --zone=us-central1-a ``` ``` kubectl get nodes ``` 1. Enable Google Container Registry (GCR) on your GCP project and configure the `docker` CLI to authenticate to GCR: ```sh gcloud services enable containerregistry.googleapis.com ``` ```sh gcloud auth configure-docker -q ``` 1. In the root of this repository, run `skaffold run --default-repo=gcr.io/[PROJECT_ID]`, where [PROJECT_ID] is your GCP project ID. This command: - builds the container images - pushes them to GCR - applies the `./kubernetes-manifests` deploying the application to Kubernetes. **Troubleshooting:** If you get "No space left on device" error on Google Cloud Shell, you can build the images on Google Cloud Build: [Enable the Cloud Build API](https://console.cloud.google.com/flows/enableapi?apiid=cloudbuild.googleapis.com), then run `skaffold run -p gcb --default-repo=gcr.io/[PROJECT_ID]` instead. 1. Find the IP address of your application, then visit the application on your browser to confirm installation. kubectl get service frontend-external **Troubleshooting:** A Kubernetes bug (will be fixed in 1.12) combined with a Skaffold [bug](https://github.com/GoogleContainerTools/skaffold/issues/887) causes load balancer to not to work even after getting an IP address. If you are seeing this, run `kubectl get service frontend-external -o=yaml | kubectl apply -f-` to trigger load balancer reconfiguration. ### Option 3: Using Pre-Built Container Images > 💡 Recommended if you want to deploy the app faster in fewer steps to an > existing cluster. **NOTE:** If you need to create a Kubernetes cluster locally or on the cloud, follow "Option 1" or "Option 2" until you reach the `skaffold run` step. This option offers you pre-built public container images that are easy to deploy by deploying the [release manifest](./release) directly to an existing cluster. **Prerequisite**: a running Kubernetes cluster (either local or on cloud). 1. Clone this repository, and go to the repository directory 1. Run `kubectl apply -f ./release/kubernetes-manifests.yaml` to deploy the app. 1. Run `kubectl get pods` to see pods are in a Ready state. 1. Find the IP address of your application, then visit the application on your browser to confirm installation. ```sh kubectl get service/frontend-external ``` ### (Optional) Deploying on a Istio-installed GKE cluster > **Note:** you followed GKE deployment steps above, run `skaffold delete` first > to delete what's deployed. 1. Create a GKE cluster (described in "Option 2"). 1. Use [Istio on GKE add-on](https://cloud.google.com/istio/docs/istio-on-gke/installing) to install Istio to your existing GKE cluster. ```sh gcloud beta container clusters update demo \ --zone=us-central1-a \ --update-addons=Istio=ENABLED \ --istio-config=auth=MTLS_PERMISSIVE ``` 2. (Optional) Enable Stackdriver Tracing/Logging with Istio Stackdriver Adapter by [following this guide](https://cloud.google.com/istio/docs/istio-on-gke/installing#enabling_tracing_and_logging). 3. Install the automatic sidecar injection (annotate the `default` namespace with the label): ```sh kubectl label namespace default istio-injection=enabled ``` 4. Apply the manifests in [`./istio-manifests`](./istio-manifests) directory. (This is required only once.) ```sh kubectl apply -f ./istio-manifests ``` 5. In the root of this repository, run `skaffold run --default-repo=gcr.io/[PROJECT_ID]`, where [PROJECT_ID] is your GCP project ID. This command: - builds the container images - pushes them to GCR - applies the `./kubernetes-manifests` deploying the application to Kubernetes. **Troubleshooting:** If you get "No space left on device" error on Google Cloud Shell, you can build the images on Google Cloud Build: [Enable the Cloud Build API](https://console.cloud.google.com/flows/enableapi?apiid=cloudbuild.googleapis.com), then run `skaffold run -p gcb --default-repo=gcr.io/[PROJECT_ID]` instead. 6. Run `kubectl get pods` to see pods are in a healthy and ready state. 7. Find the IP address of your Istio gateway Ingress or Service, and visit the application. ```sh INGRESS_HOST="$(kubectl -n istio-system get service istio-ingressgateway \ -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" echo "$INGRESS_HOST" ``` ```sh curl -v "http://$INGRESS_HOST" ``` ### Cleanup If you've deployed the application with `skaffold run` command, you can run `skaffold delete` to clean up the deployed resources. If you've deployed the application with `kubectl apply -f [...]`, you can run `kubectl delete -f [...]` with the same argument to clean up the deployed resources. ## Conferences featuring Online Boutique - [Google Cloud Next'18 London – Keynote](https://youtu.be/nIq2pkNcfEI?t=3071) showing Stackdriver Incident Response Management - Google Cloud Next'18 SF - [Day 1 Keynote](https://youtu.be/vJ9OaAqfxo4?t=2416) showing GKE On-Prem - [Day 3 – Keynote](https://youtu.be/JQPOPV_VH5w?t=815) showing Stackdriver APM (Tracing, Code Search, Profiler, Google Cloud Build) - [Introduction to Service Management with Istio](https://www.youtube.com/watch?v=wCJrdKdD6UM&feature=youtu.be&t=586) - [KubeCon EU 2019 - Reinventing Networking: A Deep Dive into Istio's Multicluster Gateways - Steve Dake, Independent](https://youtu.be/-t2BfT59zJA?t=982) --- This is not an official Google project.