PIMSimulator

1. Overview
2. HW Description
3. Setup
4. Programming Guide

1. Overview

PIMSimulator is a cycle accurate model that Single Instruction, Multiple Data (SIMD) execution units that uses the bank-level parallelism in PIM Block to boost performance that would have otherwise used multiple times of bandwidth from simultaneous access of all bank. The simulator include memory and have embedded within it a PIM block, which consist of programmable command registers, general purpose register files and execution units.

Based on https://github.com/umd-memsys/DRAMSim2, the simulator includes

PIM Block:
- Register files including CRF (for command), GRF (for vector value), SRF (for scalar value)
- ALU (ADD, MUL, MAC, MAD, MOVE, FILL, NOP, JUMP, EXIT)
PIM Kernel:
- Generate a set of memory transactions for enabling PIM operation
HBM2 support (refer to ini/HBM2_samsung_2M_16B_x64.ini)

2. HW description

PIM is a HBM stack that is pin compatible with HBM2 and have embedded within it a PIM block

2.1 Base Architecture

|--------------|          |--------------|          |--------------|
|              |    (A)   |              |   (B)    |              |
|    HOST      |----------| Controller   |----------|    Memory    |
|              |          |              |          |              |
|--------------|          |--------------|          |--------------|

Each channel is logically independent memory, so it has a dedicated independent controller.
(A): Read [Addr], Write [Addr]
(B): Activate, Read, Write, Precharge, Refresh, Activate_pim, ALU_pim, Precharge_pim, READ_pim

HBM2

System Specification: system_hbm.ini
HBM Specification: ini/HBM2_samsung_2M_16B_x64.ini
- 1 PIM block per 2 banks, 4 Bank per Bankgroup, 4 Bank group per pseudo channel, 4 pseudo channel per die, 4 die per stack.
- Prefetch size : 256bit
- burst length: 4n
- Pin speed: 2Gbps
- The simulator supports the pseudo-channel mode only, and we assume that each pseudo-channel is totally independent.

2.2 Address mapping

The address mapping is used when the memory controller decodes the address from host.
Use Scheme8 addressing mode for PIM functionality.

|<-rank->|<-row->|<-col high->|<-bg->|<-bank->|<-chan->|<-col low->|<-offset ->|

the length of col_low is log(BL * JEDEC_DATA_BUS_BUTS/8), which are 5b both for HBM2
You can also change the current addressing mode dynamically (Not recommended, though)

// Static Setting in system_*.ini
ADDRESS_MAPPING_SCHEME=Scheme8

2.3 PIM Block Placement

BANKS_PER_PIM_BLOCK = NUM_BANKS / NUM_PIM_BLOCKS

HBM2 case

|--------|  |--------|
|        |  |        |
| BANK_0 |  | BANK_2 |
|        |  |        |
|--------|  |--------|
|  PB_0  |  |  PB_1  |
|--------|  |--------|
|        |  |        |
| BANK_1 |  | BANK_3 |
|        |  |        |
|--------|  |--------|

if 2 * NUM_PIM_BLOCKS == NUM_BANK, a PIM block is located per two banks.
- NUM_PIM_BLOCKS = 8, NUM_BANKS = 16
if NUM_PIM_BLOCKS == NUM_BANKS, a PIM Block (PB) is located per banks.
- NUM_PIM_BLOCKS = 8, NUM_BANKS = 8

2.3 PIM Instruction-Set Architecture

Type	Command	Description	Result (DST)	Operand (SRC0)	Operand (SRC1)
Arithmetic	ADD	addition	GRF	GRF, BANK, SRF	GRF, BANK, SRF
Arithmetic	MUL	multiplication	GRF	GRF, BANK	GRF, BANK, SRF
Arithmetic	MAC	multiply-accumulate	GRF_B	GRF, BANK	GRF, BANK, SRF
Arithmetic	MAD	multiply-and-add	GRF	GRF, BANK	GRF, BANK, SRF
Data	MOV	load or store data from register to bank	GRF, SRF	GRF, BANK
Data	FILL	copy data from bank to register	GRF, BANK	GRF, BANK
Control	NOP	do nothing
Control	JUMP	jump instruction
Control	EXIT	exit instruction

Supports RISC-style 32-bit instructions
Three instructions types
- 4 Arithmetic: ADD, MUL, MAC, MAD
- 2 Data transfer: MOV, FILL
- 3 control flows: NOP, JUMP, EXIT
JUMP instruction
- Zero-cycle static branch: supports only a pre-programmed numbers of iterations
Operand type:
- Vector Register (GRF_A, GRF_B)
- Scalar Register (SRF)
- Bank Row Buffer
PIM instructions are stored in the Command Register File (CRF), and memory command triggers a CRF to perform a target instruction
- each memory command increments the CRF PC
DRAM commands decide where to retrieve data from DRAM for PIM arithmetic operations

2.4 Movement of Data

Mode	Transaction	PIM Instruction	Operation
SB	Read	-	Normal Memory Read
SB	Write	-	Normal Memory Write
HAB	Write	-	PIM Write (Host to PIM Register)
PIM	-	MOV	read or write from bank to PIM Register
PIM	-	FILL	write from bank to PIM Registers

SB mode: standard DRAM operation
HAB mode: Allowing concurrent accesses to multiple banks with a single DRAM command
PIM mode: Triggers the execution of PIM commands on the CRF by DRAM Command

3. Setup

3.1 Prerequisites

Scons tool for compiling PIMSimulator:

sudo apt install scons

gtest for running test cases:

sudo apt install libgtest-dev

sudo apt install libgtest-dev cmake
cd /usr/src/gtest
sudo cmake CMakeLists.txt
sudo make
sudo cp *.a /usr/lib

3.2 Installing

To Install PIMSimulator:

# compile
scons

3.3 Launch a Test Run

Show a list of test cases

./sim --gtest_list_tests

# Example
PIMKernelFixture.
  gemv_tree
  gemv
  mul
  add
  relu
MemBandwidthFixture.
  hbm_read_bandwidth
  hbm_write_bandwidth
PIMBenchFixture.
  gemv
  mul
  add
  relu

Test Running

# Running: functionality test (GEMV)
./sim --gtest_filter=PIMKernelFixture.gemv

# Running: functionality test (MUL)
./sim --gtest_filter=PIMKernelFixture.mul

# Running: performance test (GEMV)
./sim --gtest_filter=PIMBenchFixture.gemv

# Running: performance test (ADD)
./sim --gtest_filter=PIMBenchFixture.add

If you want to functionality test for other dimensions, generate a new dimension in ./data and add generated dimension to the source of src/tests/KernelTestCases.cpp. Use the gen script in ./data to generate data of the dimension to be changed.

3.4 Configuration

Turning on/off verbose mode

You can select what kinds of log you want to see by modifying system_*.ini

Turning on/off data mode

Data mode
- build without -DNO_STORAGE option
No-data mode
- build with -DNO_STORAGE option

# build to No-data mode
scons NO_STORAGE=1

4 Programming Guide

Highly recommend you to refer to src/tests/* (especially, src/tests/PIMKernel.cpp and src/tests/PIMBenchTestCases.cpp) To attach to host simulator, refer to src/tests/PIMKernel.cpp. You can see commands that request memory transactions to the memory controller for GEMV or Eltwise operations on PIM. It include a basic PIM procedure for GEMV operation in the PIMKernel::executeGemv(), and also for Eltwise operation (add, mul, relu) in the PIMKernel::executeEltwise()

4.1 Primitive Function

mem->addTransaction(is_read, address, tag, buffer);

is_read: memory request types between READ('false') and WRITE('true')
address: address used for memory / PIM transaction
tag: Used for log or set to barrier. If not used, only three parameters are available, as addTransaction(is_read, address, buffer)
buffer : used to verify pim functionality using data. Here, the buffer is at least 256-bit sized container. If you do not want to use the data buffer, you can use it as below:
```
BurstType nullBst
mem->addTransaction(isWrite, addr, &nullBst);
```

Memory transaction

read

mem->addTransaction(false, addr, tag, buffer);

write

mem->addTransaction(true, addr, tag, buffer);

Here, the buffer must be at least 256bit size container.

PIM transaction

alu_pim (dataflow is similar to normal write)
```
mem->addTransaction(true, addr, tag, buffer);
```
- Highly recommend you to refer to simple PIM operations using PIM ISA (src/tests/PIMCmdGen.h) and procedures using them(src/tests/PIMKernel.cpp)
- The buffer must contain data to be broadcasted to all pim blocks of a specific channel.
- In GEMV cacse, the corresponding weight is supplied from specific row, a specific col of multiple banks of a specific memory channel.
  - If each bank in a channel has unique ID, and bank addr in the transaction is BA, the banks satisfying (ID % BANKS_PER_PIM_BLOCK == BA) supply the weight to PIM blocks.
    - if BANKS_PER_PIM_BLOCK == 2, and BA = 0, bank 0,2,4,6,... supply the weight to pim-block 0,1,2,3,..., respectively.
    - if BANKS_PER_PIM_BLOCK == 2, and BA = 1, bank 1,3,5,7,... supply the weight to pim-block 0,1,2,3,..., respectively.
  - As a result, data broadcasted and data from multiple banks of a specific channel are multiplied and accumulated.
read_pim (dataflow is similar to normal read)
```
mem->addTransaction(false, addr, tag, buffer);
```
- Read the accumulated partial sum in the pim block. Then reset the buffer.
  - If each pim block in a channel has unique ID, and bank addr in the transaction is BA, the PIM block satisfying (ID == BA) supply partial sums to DQ.
- Highly recommend to use the address that alu_pim command used at the last

4.2 PIM High-level Steps

The following shows the high level steps of a generic PIM operation.
- Place data in DRAM
- Switch to HAB mode
- Program CRF
- Enable PIM
- Execute PIM
- Disable PIM
- Switch to SB mode
A similar procedure at the source level can be found in src/tests/PIMKernel.cpp.

    /* Example Code - PIMKernel::executeELtwise() */

    parkIn();
    changePIMMode(dramMode::SB, dramMode::HAB);      // Switch to HAB
    programCrf(pim_cmds);                            // Program CRF
    changePIMMode(dramMode::HAB, dramMode::HAB_PIM); // Enable PIM

    if (ktype == KernelType::ADD || ktype == KernelType::MUL)
        computeAddOrMul(num_tile, input0_row, result_row, input1_row); // Execute PIM
    else if (ktype == KernelType::RELU)
        computeRelu(num_tile, input0_row, result_row);

    changePIMMode(dramMode::HAB_PIM, dramMode::HAB); // Disable PIM
    changePIMMode(dramMode::HAB, dramMode::SB);      // Switch to SB mode
    parkOut();

The other basic operation flow on PIM for GEMV(Matrix Vector multiplication), Element-wise operation are described in the src/tests/PIMKernel.cpp.

Contact

Shin-haeng Kang (s-h.kang@samsung.com)
Sanghoon Cha (s.h.cha@samsung.com)
Seungwoo Seo (sgwoo.seo@samsung.com)
Jin-seong kim (jseong82.kim@samsung.com)

akothen/PIMSimulator