project-gulimall-backend: A JavaScript repository from codingEzio

Prep

macOS

Tools

With the help of

Docker Port-mapping (.. run -p3306:3306 ..)
Vagrantfile (config.vm.network "forwarded_port", guest:3306,host: 3306)

You can connect to them using localhost:3306 like they are in the host machine

If you also have an app which uses port 3336, just change the host: PORT in Vagrantfile

To make the MySQL and Redis to run when you boot up the VM, add

docker update mysql5dot7 --restart=always
docker update redis6dot16 --restart=always

The Foundation Vagrant

Almost all the tools were installed to the VM, including Docker

Get started

brew install vagrant    # use the GUI way if you prefer

vagrant --version       # check if installed correctly

vagrant init centos/7   # config file

# cd the where the Vagrantfile file lies in
vagrant up              # download, config and boot
vagrant ssh             # ssh into the virtual machine

Networking

To make the VM like just another physical PC in your LAN

# Edit the Vagrantfile
config.vm.network "public_network"

# Run `vagrant up`
# You shall choose the one you used to connect to the Internet

Docker

Run under the user root

# 1. Remove existing docker just in case any problems arise
# https://docs.docker.com/engine/install/centos/#uninstall-old-versions
yum remove docker-engine \
    docker-common \
    docker docker-latest \
    docker-client docker-client-latest \
    docker-logrotate docker-latest-logrotate  \


# 2. Let yum know where to find Docker to install
# https://docs.docker.com/engine/install/centos/#install-using-the-repository

yum install -y yum-utils
yum-config-manager \
    --add-repo \
    "https://download.docker.com/linux/centos/docker-ce.repo"

yum install docker-ce docker-ce-cli containerd.io docker-compose-plugin


# 3. Configure mirrors Docker images for faster fetching
# https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors
mkdir -p /etc/docker
touch    /etc/docker/daemon.json
echo '{"registry-mirrors": ["https://22diassb.mirror.aliyuncs.com"]}' > /etc/docker/daemon.json


# 4. Configure Docker to run when the OS boots
systemctl enable docker    # add to 'run when OS boots' list


# 5. Run
systemctl start docker

MySQL

Run under the user root

Get started

# 1. Get MySQL
docker pull mysql:5.7
docker images | grep mysql

docker container ls
docker container stop mysql5dot7
docker container rm   mysql5dot7


# 2. Run MySQL
docker run -p 3306:3306 --name mysql5dot7 \
    -v /mydata/mysql/log:/var/log/mysql \
    -v /mydata/mysql/data:/var/lib/mysql \
    -v /mydata/mysql/conf:/etc/mysql \
    -e MYSQL_ROOT_PASSWORD=root \
    -d mysql:5.7

Configuration

Ready to edit the config for MySQL 5.7 in VM (<- Docker container)

# [VIRTUAL MACHINE]     [DOCKER CONTAINER]
# /mydata/mysql/log     /var/log/mysql
# /mydata/mysql/data    /var/lib/mysql
# /mydata/mysql/conf    /etc/mysql

Edit MySQL config ( sudo vi /mydata/mysql/conf/my.cnf )

[client]
default-character-set=utf8

[mysql]
default-character-set=utf8

[mysqld]
init_connect='SET NAMES utf8'
init_connect='SET collation_connection = utf8_unicode_ci'

character-set-server=utf8
collation-server=utf8_unicode_ci

skip-character-set-client-handshake
skip-name-resolve

Reload new configuration

docker restart mysql5dot7

Check new configuration

docker exec -it mysql5dot7 /bin/bash     # VM
cat /etc/mysql/my.cnf                    # MySQL container

Redis

Run under the user root

Get started

# 1. Get Redis
docker pull redis:6.0.16
docker images | grep redis

docker container ls
docker container stop redis6dot16
docker container rm   redis6dot16


# 2. Initialization before running
# Mapping the config in VM into the Redis container
mkdir -p /mydata/redis/conf
touch /mydata/redis/conf/redis.conf


# 3. Run Redis
docker run -p 6379:6379 --name redis6dot16 \
    -v /mydata/redis/data:/data \
    -v /mydata/redis/conf/redis.conf:/etc/redis/redis.conf \
    -d redis:6.0.16 \
    redis-server /etc/redis/redis.conf

Configuration

# 1. Edit configuration (Redis persistence)
echo 'appendonly yes' > /mydata/redis/conf/redis.conf


# 2. Check configuration
docker exec -it redis6dot16 /bin/bash    # VM
cat /etc/redis/redis.conf                # Redis container

Maven

AliYun mirror (for faster download) and use JDK 1.8 to compile
Editors

Plugins to install
- IDEA: Lombok, MyBatisX
- VS Code: whatever suits you

Data

I'm using mycli (mycli -u root -h localhost) and BeeKeeper to manage the databases

Create the database

Reference for the command, the why for utf8mb4 and utf8mb4_unicode_ci

CREATE DATABASE IF NOT EXISTS gulimall_pms_product
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

CREATE DATABASE IF NOT EXISTS gulimall_oms_order
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

CREATE DATABASE IF NOT EXISTS gulimall_sms_salepromo
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

CREATE DATABASE IF NOT EXISTS gulimall_ums_user
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

CREATE DATABASE IF NOT EXISTS gulimall_wms_logistics
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

Load the schema

cd to where the .sql files are and get mycli installed (reference to source)

USE    gulimall_pms_product       ;
source gulimall_pms_product.sql   ;

USE    gulimall_oms_order         ;
source gulimall_oms_order.sql     ;

USE    gulimall_sms_salepromo     ;
source gulimall_sms_salepromo.sql ;

USE    gulimall_ums_user          ;
source gulimall_ums_user.sql      ;

USE    gulimall_wms_logistics     ;
source gulimall_wms_logistics.sql ;

Scaffold

Remove the .git folder before you include or start using these

Backend

Backend: renren-fast

Create the database

CREATE DATABASE IF NOT EXISTS gulimall_admin_renrenfast
    DEFAULT CHARACTER SET utf8mb4
    DEFAULT COLLATE utf8mb4_unicode_ci ;

Load the data from the .sql included in the repository

cd to where the .sql files are and get mycli installed (reference to source)
```
use    gulimall_admin_renrenfast     ;
source gulimall_admin_renrenfast.sql ;
```

Run

mvn clean
mvn install

# Now you can run the start up the frontend Vue.js project
mvn spring-boot:run

Frontend

Frontend: renren-fast-vue

Make sure you already have the backend server running

yarn install

# 1. The name and the password are both 'admin'
# 2. The captcha was generated by the backend server
yarn run dev

Code Generation

Get renren-generator and fix basic dep issues

Snippet

Component 👐 Database

/*
gulimall-coupon     gulimall_sms_salepromo
gulimall-member     gulimall_ums_user
gulimall-order      gulimall_oms_order
gulimall-product    gulimall_pms_product
gulimall-ware       gulimall_wms_logistics
*/

Procedure

Make copies based on the template

ORIG_CONF_APP='template.application.yml'
ORIG_CONF_GEN='template.generator.properties'

# Then edit the details by your own
for f in application-{coupon,member,order,product,ware}.yml;
    do cp ${ORIG_CONF_APP} $f;
done

# Then edit the details by your own
for f in generator-{coupon,member,order,product,ware}.properties;
    do cp ${ORIG_CONF_GEN} $f;
done

Run the generation service

Modify the directory in correspondence with yours

COMP="coupon"
PROJ_ROOT="/Users/mac/dev/ytb-projects-gulimall"

cd "${PROJ_ROOT}/renren-generator/src/main/resources/"
cp -fv application-${COMP}.yml application.yml ;
cp -fv generator-${COMP}.properties generator.properties ;

cd "${PROJ_ROOT}/renren-generator"
mvn clean install && mvn spring-boot:run

cd "${PROJ_ROOT}"

Start the generation

PORT=80
open "http://localhost:${PORT}#generator.html"

Add the generated code to our components

Make sure you haven't written anything new to the components!

cd "/Users/mac/dev/ytb-projects-gulimall"

for comp in {coupon,member,order,product,ware}.zip;
    do
    unzip \
        "dev_generatedcode/srcmain-gulimall-${comp}.zip" \
        -d "gulimall-${comp}/src/" &> /dev/null
done

Fix dependencies

Create dedicated package to hold common dependencies

The main theme is copying files around and mvn clean install

cd "/Users/mac/dev/ytb-projects-gulimall"

mkdir -p \
    gulimall-common \
    gulimall-common/com/elliot/common/{utils,xss}/

pom.xml for gulimall-common

<?xml version="1.0" encoding="UTF-8"?>
<project .. >

    <parent>
        <artifactId>gulimall</artifactId>
        <groupId>com.elliot.gulimall</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.elliot.gulimall</groupId>
    <artifactId>gulimall-common</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>gulimall-common</name>
    <description>项目共用依赖</description>

    <dependencies>
        ...
    </dependencies>

</project>

Modify pom.xml in other components for them to use common dependencies

<!-- Add right after the starting section of <dependencies> -->
..
    <dependencies>
        <dependency>
            <groupId>com.elliot.gulimall</groupId>
            <artifactId>gulimall-common</artifactId>
            <version>0.0.1-SNAPSHOT</version>
        </dependency>
    </dependencies>
..

Microservice in Practice

For non-microservices, only three parts were required, the frondend, the database and the backend. But when the business grows larger and more complex, you would need an efficient and effective way to manage the scale and complexity.

And the backend is where we targeting on, to make it not just scalable, but also manageable. Technologies which were provided by Spring Cloud or Spring Cloud Alibaba is the set of tools which helps us to to do that (scalable and manageable).

Don't be scared by the terminologies

Names like Service Registration: the name of the solution for the problems we would face (in this context: letting different services find each other)
Names like Nacos: the name of a tool which delivers the solutions for at least one of the problems you would face

Nacos

Solution
- Service Registration
- Service Discovery
- Load Balancing (just the setup in term of progress)
- Configuration Management
Usage Overview

A server running at the background plus a few registered services

Service Discovery & Registration

Setup for the Server

Download the Nacos server

Do not put these under your version control, which would exceed the 100MB limit, as you need to handle the Git-LFS related issues

# Download
wget \
    -O nacos-server-2.1.1.zip \
    https://github.com/alibaba/nacos/releases/download/2.1.1/nacos-server-2.1.1.zip


# Extract
unzip nacos-server-2.1.1.zip -d .

Run

# Start
bash ./nacos/bin/startup.sh -m standalone


# Check if it started correctly
cat ./nacos/logs/start.out

Setup for the Client

Download the dependency

Add this to the pom.xml inside the gulimall-common package

<!-- Right after where the level 'mysql-connector-java' is in -->
<!-- Other components would be able to use this as well! -->
..
    <dependency>
        <dependency>
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
    </dependency>
..

Configuration

Add this to whichever components you wanna test on

# application.yml
spring:
  application:
    name: gulimall-COMPONENT
  ..
    cloud:
      nacos:
        discovery:
          server-addr=127.0.0.1:8848

Annotation

Add this to whichever components you wanna test on

// src/main/java -> PACKAGE -> Gulimall 👉COMPONENT👈 Application.java
import .. ;
import .. ;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;

@ ..
@ ..
@EnableDiscoveryClient
public class GulimallCouponApplication { .. }

Get the clients running (registering)

# Go to the root folder of whichever components you wanna test on
./mvnw spring-boot:run


# Now you could go to localhost:8848 to check if they registered correctly
# Both the username and the password are 'nacos' (lowercase)

Load Balancing

Basically we are choosing the one included in Nacos instead of Netflix Ribbon

Add this to the pom.xml in the gulimall-common so all the components could use it

..
    <dependency>
        <!-- Already added this, showing you where to add '<exclusion>' tag -->
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>

        <!-- Excluding this Load Balancer, we'll use the one down below -->
        <exclusions>
            <exclusion>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-netflix-ribbon</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        <version>2.2.9.RELEASE</version>
    </dependency>
..

Configuration Management

Basically we'll let components be able to send a (HTTP/RPC) request to upload the configurations

Configuration

Followed these steps from the Nacos documentation

Add this to the pom.xml in the gulimall-common so all the components are able to use them

..
    <!-- Nacos: Configuration Management -->
    <dependency>
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
    </dependency>


    <!-- Dedicated for bootstrap.properties which Nacos use it to configure -->
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-bootstrap</artifactId>
        <version>3.1.4</version>
    </dependency>

Config file bootstrap.properties for Nacos configuration management

Nacos requires this to configure the config management (their doc references it)
```
touch \
    gulimall-{coupon,member,order,product,ware}/src/main/resources/bootstrap.yml
```

Configuration for/with bootstrap.yml

Two files were your concern: application.yml and the bootstrap.yml

Enable bootstrap

# application.yml

spring:
  cloud:
    bootstrap:
      enabled: true

Configure bootstrap

Modify the name and the prefix for each components as needed

# bootstrap.yml

spring:
  application:
    # Should be the same as the one you defined in application.yml
    name: nacos-gulimall-coupon
  cloud:
    nacos:
      config:
        server-addr: 127.0.0.1:8848

        # Combining these two, you get the data ID for you to add to the
        # Nacos configuration center for hot-reloading config, eventually,
        # you can put 'nacosconfig-coupon.yaml' in the configuration server
        # for future update
        prefix: nacosconfig-coupon
        file-extension: yaml

Setup for Hot-reload

Without this, the procedure for updating the configuration is edit config, deploy then send request to see the changes being made. BUT! If you have multiple machines (each with their own different config), you would have to do the same process over and over!

Code which reads from the config file

The @Value here is used either for reading config keys ${} or templates #{}

import .. ;
import .. ;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.context.config.annotation.RefreshScope;

@ ..
@ ..
@RefreshScope
public class CouponController {

    // We gotta set a default value for the keys here, since some of the
    // config would only be available when the configuration server is
    // online (at least that's my experience: errors being raised by
    // Spring Boot when I'm not setting up the default value).

    @Value("${coupon.user.name:nobody}")  // default for name: 'nobody'
    private String name;

    @Value("${coupon.user.age:1}")        // default for age: 1
    private String age;

    // Test if it would read the contents from the config file
    @RequestMapping("/test")
    public R test() {
        return R.ok()
                .put("name", name)
                .put("age", age);
    }
}

OpenFeign

Solution
- A way for components calling each other (HTTP requests, but enhanced)
Usage Overview

N/A

Configuration

Add this to the pom.xml under your individual components' folder

..
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-openfeign</artifactId>
    </dependency>
..

Components to be called

gulimall-coupon (as for gulimallcoupon, it's the lowest level package's name)

A new method inside CouponController.java

@ ..
@RequestMapping("gulimallcoupon/coupon")
public class CouponController {
    @Autowired
    private CouponService couponService;

    // This is what we added, any other code exist already
    @RequestMapping("/member/list")
    public R membercoupons() {
        CouponEntity couponEntity = new CouponEntity();
        couponEntity.setCouponName("30% off");

        return R.ok().put("coupons", Arrays.asList(couponEntity));
    }
}

Components who do the calling

gulimall-member

A central place to put available calling methods for the caller

cd './gulimall-member/ .. /com/elliot/gulimall/gulimallmember/'

mkdir -p feign

Where to find the callers

import org.springframework.cloud.openfeign.EnableFeignClients;

@ ..
@ ..
@EnableFeignClients(basePackages = "com.elliot.gulimall.gulimallmember.feign")
public class GulimallMemberApplication { .. }

Write an interface

import com.elliot.common.utils.R;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.RequestMapping;

@FeignClient("gulimallcoupon")
public interface CouponFeignService {

    @RequestMapping("gulimallcoupon/coupon/member/list")
    public R membercoupons();
}

`gulimallmember` Calls `gulimallcoupon`

Test code

@ ..
@RequestMapping("gulimallmember/member")
public class MemberController {
    ..

    @Autowired
    private CouponFeignService couponFeignService;

    @RequestMapping("/coupons")
    public R test() {
        MemberEntity memberEntity = new MemberEntity();
        memberEntity.setNickname("Dickson");

        R membercoupons = couponFeignService.membercoupons();

        return R.ok()
                .put("member", memberEntity)
                .put("coupons", membercoupons.get("coupons"));
    }

Check the result

PORT_MEMBER=8000
ENDPOINT="http://localhost:${PORT_MEMBER}/gulimallmember/member/coupons"

curl "${ENDPOINT}" | jq '.coupons [] .couponName'    # "30% off"
curl "${ENDPOINT}" | jq '.member .nickname'          # "Dickson"

References

Tools

Vagrant

vagrant status

vagrant suspend     # sleep
vagrant up          # boot

vagrant halt        # shutdown
vagrant reload      # reboot

Docker

docker container ps -a
docker container restart mysql5dot7

docker exec -it mysql5dot7 /bin/bash

MySQL

-- Just in case you wanna start over
DROP DATABASE gulimall_pms_product   ;
DROP DATABASE gulimall_oms_order     ;
DROP DATABASE gulimall_sms_salepromo ;
DROP DATABASE gulimall_ums_user      ;
DROP DATABASE gulimall_wms_logistics ;

codingEzio/project-gulimall-backend

Prep

Tools

Data

Scaffold

Backend

Frontend

Code Generation

Snippet

Procedure

Microservice in Practice

Nacos

Service Discovery & Registration

Load Balancing

Configuration Management

Configuration

Setup for Hot-reload

OpenFeign

Configuration

Components to be called

Components who do the calling

gulimallmember Calls gulimallcoupon

References

Tools

`gulimallmember` Calls `gulimallcoupon`