Repository: adam-maj/tiny-gpu
Branch: master
Commit: 02b6c2ce223f
Files: 25
Total size: 32.4 MB
Directory structure:
gitextract_a16b4xwm/
├── .gitignore
├── Makefile
├── README.md
├── gds/
│ ├── 0/
│ │ └── gpu.gds
│ └── 1/
│ └── gpu.gds
├── src/
│ ├── alu.sv
│ ├── controller.sv
│ ├── core.sv
│ ├── dcr.sv
│ ├── decoder.sv
│ ├── dispatch.sv
│ ├── fetcher.sv
│ ├── gpu.sv
│ ├── lsu.sv
│ ├── pc.sv
│ ├── registers.sv
│ └── scheduler.sv
└── test/
├── __init__.py
├── helpers/
│ ├── format.py
│ ├── logger.py
│ ├── memory.py
│ └── setup.py
├── logs/
│ └── .gitkeep
├── test_matadd.py
└── test_matmul.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*/**/__pycache__
build/
test/logs/*
!.gitkeep
gds/**/*.gltf
.DS_Store
results.xml
================================================
FILE: Makefile
================================================
.PHONY: test compile
export LIBPYTHON_LOC=$(shell cocotb-config --libpython)
test_%:
make compile
iverilog -o build/sim.vvp -s gpu -g2012 build/gpu.v
MODULE=test.test_$* vvp -M $$(cocotb-config --prefix)/cocotb/libs -m libcocotbvpi_icarus build/sim.vvp
compile:
make compile_alu
sv2v -I src/* -w build/gpu.v
echo "" >> build/gpu.v
cat build/alu.v >> build/gpu.v
echo '`timescale 1ns/1ns' > build/temp.v
cat build/gpu.v >> build/temp.v
mv build/temp.v build/gpu.v
compile_%:
sv2v -w build/$*.v src/$*.sv
# TODO: Get gtkwave visualizaiton
show_%: %.vcd %.gtkw
gtkwave $^
================================================
FILE: README.md
================================================
# tiny-gpu
A minimal GPU implementation in Verilog optimized for learning about how GPUs work from the ground up.
Built with <15 files of fully documented Verilog, complete documentation on architecture & ISA, working matrix addition/multiplication kernels, and full support for kernel simulation & execution traces.
### Table of Contents
- [Overview](#overview)
- [Architecture](#architecture)
- [GPU](#gpu)
- [Memory](#memory)
- [Core](#core)
- [ISA](#isa)
- [Execution](#execution)
- [Core](#core-1)
- [Thread](#thread)
- [Kernels](#kernels)
- [Matrix Addition](#matrix-addition)
- [Matrix Multiplication](/tree/master?tab=readme-ov-file#matrix-multiplication)
- [Simulation](#simulation)
- [Advanced Functionality](#advanced-functionality)
- [Next Steps](#next-steps)
# Overview
If you want to learn how a CPU works all the way from architecture to control signals, there are many resources online to help you.
GPUs are not the same.
Because the GPU market is so competitive, low-level technical details for all modern architectures remain proprietary.
While there are lots of resources to learn about GPU programming, there's almost nothing available to learn about how GPU's work at a hardware level.
The best option is to go through open-source GPU implementations like [Miaow](https://github.com/VerticalResearchGroup/miaow) and [VeriGPU](https://github.com/hughperkins/VeriGPU/tree/main) and try to figure out what's going on. This is challenging since these projects aim at being feature complete and functional, so they're quite complex.
This is why I built `tiny-gpu`!
## What is tiny-gpu?
> [!IMPORTANT]
>
> **tiny-gpu** is a minimal GPU implementation optimized for learning about how GPUs work from the ground up.
>
> Specifically, with the trend toward general-purpose GPUs (GPGPUs) and ML-accelerators like Google's TPU, tiny-gpu focuses on highlighting the general principles of all of these architectures, rather than on the details of graphics-specific hardware.
With this motivation in mind, we can simplify GPUs by cutting out the majority of complexity involved with building a production-grade graphics card, and focus on the core elements that are critical to all of these modern hardware accelerators.
This project is primarily focused on exploring:
1. **Architecture** - What does the architecture of a GPU look like? What are the most important elements?
2. **Parallelization** - How is the SIMD progamming model implemented in hardware?
3. **Memory** - How does a GPU work around the constraints of limited memory bandwidth?
After understanding the fundamentals laid out in this project, you can checkout the [advanced functionality section](#advanced-functionality) to understand some of the most important optimizations made in production grade GPUs (that are more challenging to implement) which improve performance.
# Architecture
<p float="left">
<img src="/docs/images/gpu.png" alt="GPU" width="48%">
<img src="/docs/images/core.png" alt="Core" width="48%">
</p>
## GPU
tiny-gpu is built to execute a single kernel at a time.
In order to launch a kernel, we need to do the following:
1. Load global program memory with the kernel code
2. Load data memory with the necessary data
3. Specify the number of threads to launch in the device control register
4. Launch the kernel by setting the start signal to high.
The GPU itself consists of the following units:
1. Device control register
2. Dispatcher
3. Variable number of compute cores
4. Memory controllers for data memory & program memory
5. Cache
### Device Control Register
The device control register usually stores metadata specifying how kernels should be executed on the GPU.
In this case, the device control register just stores the `thread_count` - the total number of threads to launch for the active kernel.
### Dispatcher
Once a kernel is launched, the dispatcher is the unit that actually manages the distribution of threads to different compute cores.
The dispatcher organizes threads into groups that can be executed in parallel on a single core called **blocks** and sends these blocks off to be processed by available cores.
Once all blocks have been processed, the dispatcher reports back that the kernel execution is done.
## Memory
The GPU is built to interface with an external global memory. Here, data memory and program memory are separated out for simplicity.
### Global Memory
tiny-gpu data memory has the following specifications:
- 8 bit addressability (256 total rows of data memory)
- 8 bit data (stores values of <256 for each row)
tiny-gpu program memory has the following specifications:
- 8 bit addressability (256 rows of program memory)
- 16 bit data (each instruction is 16 bits as specified by the ISA)
### Memory Controllers
Global memory has fixed read/write bandwidth, but there may be far more incoming requests across all cores to access data from memory than the external memory is actually able to handle.
The memory controllers keep track of all the outgoing requests to memory from the compute cores, throttle requests based on actual external memory bandwidth, and relay responses from external memory back to the proper resources.
Each memory controller has a fixed number of channels based on the bandwidth of global memory.
### Cache (WIP)
The same data is often requested from global memory by multiple cores. Constantly access global memory repeatedly is expensive, and since the data has already been fetched once, it would be more efficient to store it on device in SRAM to be retrieved much quicker on later requests.
This is exactly what the cache is used for. Data retrieved from external memory is stored in cache and can be retrieved from there on later requests, freeing up memory bandwidth to be used for new data.
## Core
Each core has a number of compute resources, often built around a certain number of threads it can support. In order to maximize parallelization, these resources need to be managed optimally to maximize resource utilization.
In this simplified GPU, each core processed one **block** at a time, and for each thread in a block, the core has a dedicated ALU, LSU, PC, and register file. Managing the execution of thread instructions on these resources is one of the most challening problems in GPUs.
### Scheduler
Each core has a single scheduler that manages the execution of threads.
The tiny-gpu scheduler executes instructions for a single block to completion before picking up a new block, and it executes instructions for all threads in-sync and sequentially.
In more advanced schedulers, techniques like **pipelining** are used to stream the execution of multiple instructions subsequent instructions to maximize resource utilization before previous instructions are fully complete. Additionally, **warp scheduling** can be use to execute multiple batches of threads within a block in parallel.
The main constraint the scheduler has to work around is the latency associated with loading & storing data from global memory. While most instructions can be executed synchronously, these load-store operations are asynchronous, meaning the rest of the instruction execution has to be built around these long wait times.
### Fetcher
Asynchronously fetches the instruction at the current program counter from program memory (most should actually be fetching from cache after a single block is executed).
### Decoder
Decodes the fetched instruction into control signals for thread execution.
### Register Files
Each thread has it's own dedicated set of register files. The register files hold the data that each thread is performing computations on, which enables the same-instruction multiple-data (SIMD) pattern.
Importantly, each register file contains a few read-only registers holding data about the current block & thread being executed locally, enabling kernels to be executed with different data based on the local thread id.
### ALUs
Dedicated arithmetic-logic unit for each thread to perform computations. Handles the `ADD`, `SUB`, `MUL`, `DIV` arithmetic instructions.
Also handles the `CMP` comparison instruction which actually outputs whether the result of the difference between two registers is negative, zero or positive - and stores the result in the `NZP` register in the PC unit.
### LSUs
Dedicated load-store unit for each thread to access global data memory.
Handles the `LDR` & `STR` instructions - and handles async wait times for memory requests to be processed and relayed by the memory controller.
### PCs
Dedicated program-counter for each unit to determine the next instructions to execute on each thread.
By default, the PC increments by 1 after every instruction.
With the `BRnzp` instruction, the NZP register checks to see if the NZP register (set by a previous `CMP` instruction) matches some case - and if it does, it will branch to a specific line of program memory. _This is how loops and conditionals are implemented._
Since threads are processed in parallel, tiny-gpu assumes that all threads "converge" to the same program counter after each instruction - which is a naive assumption for the sake of simplicity.
In real GPUs, individual threads can branch to different PCs, causing **branch divergence** where a group of threads threads initially being processed together has to split out into separate execution.
# ISA
tiny-gpu implements a simple 11 instruction ISA built to enable simple kernels for proof-of-concept like matrix addition & matrix multiplication (implementation further down on this page).
For these purposes, it supports the following instructions:
- `BRnzp` - Branch instruction to jump to another line of program memory if the NZP register matches the `nzp` condition in the instruction.
- `CMP` - Compare the value of two registers and store the result in the NZP register to use for a later `BRnzp` instruction.
- `ADD`, `SUB`, `MUL`, `DIV` - Basic arithmetic operations to enable tensor math.
- `LDR` - Load data from global memory.
- `STR` - Store data into global memory.
- `CONST` - Load a constant value into a register.
- `RET` - Signal that the current thread has reached the end of execution.
Each register is specified by 4 bits, meaning that there are 16 total registers. The first 13 register `R0` - `R12` are free registers that support read/write. The last 3 registers are special read-only registers used to supply the `%blockIdx`, `%blockDim`, and `%threadIdx` critical to SIMD.
# Execution
### Core
Each core follows the following control flow going through different stages to execute each instruction:
1. `FETCH` - Fetch the next instruction at current program counter from program memory.
2. `DECODE` - Decode the instruction into control signals.
3. `REQUEST` - Request data from global memory if necessary (if `LDR` or `STR` instruction).
4. `WAIT` - Wait for data from global memory if applicable.
5. `EXECUTE` - Execute any computations on data.
6. `UPDATE` - Update register files and NZP register.
The control flow is laid out like this for the sake of simplicity and understandability.
In practice, several of these steps could be compressed to be optimize processing times, and the GPU could also use **pipelining** to stream and coordinate the execution of many instructions on a cores resources without waiting for previous instructions to finish.
### Thread
Each thread within each core follows the above execution path to perform computations on the data in it's dedicated register file.
This resembles a standard CPU diagram, and is quite similar in functionality as well. The main difference is that the `%blockIdx`, `%blockDim`, and `%threadIdx` values lie in the read-only registers for each thread, enabling SIMD functionality.
# Kernels
I wrote a matrix addition and matrix multiplication kernel using my ISA as a proof of concept to demonstrate SIMD programming and execution with my GPU. The test files in this repository are capable of fully simulating the execution of these kernels on the GPU, producing data memory states and a complete execution trace.
### Matrix Addition
This matrix addition kernel adds two 1 x 8 matrices by performing 8 element wise additions in separate threads.
This demonstration makes use of the `%blockIdx`, `%blockDim`, and `%threadIdx` registers to show SIMD programming on this GPU. It also uses the `LDR` and `STR` instructions which require async memory management.
`matadd.asm`
```asm
.threads 8
.data 0 1 2 3 4 5 6 7 ; matrix A (1 x 8)
.data 0 1 2 3 4 5 6 7 ; matrix B (1 x 8)
MUL R0, %blockIdx, %blockDim
ADD R0, R0, %threadIdx ; i = blockIdx * blockDim + threadIdx
CONST R1, #0 ; baseA (matrix A base address)
CONST R2, #8 ; baseB (matrix B base address)
CONST R3, #16 ; baseC (matrix C base address)
ADD R4, R1, R0 ; addr(A[i]) = baseA + i
LDR R4, R4 ; load A[i] from global memory
ADD R5, R2, R0 ; addr(B[i]) = baseB + i
LDR R5, R5 ; load B[i] from global memory
ADD R6, R4, R5 ; C[i] = A[i] + B[i]
ADD R7, R3, R0 ; addr(C[i]) = baseC + i
STR R7, R6 ; store C[i] in global memory
RET ; end of kernel
```
### Matrix Multiplication
The matrix multiplication kernel multiplies two 2x2 matrices. It performs element wise calculation of the dot product of the relevant row and column and uses the `CMP` and `BRnzp` instructions to demonstrate branching within the threads (notably, all branches converge so this kernel works on the current tiny-gpu implementation).
`matmul.asm`
```asm
.threads 4
.data 1 2 3 4 ; matrix A (2 x 2)
.data 1 2 3 4 ; matrix B (2 x 2)
MUL R0, %blockIdx, %blockDim
ADD R0, R0, %threadIdx ; i = blockIdx * blockDim + threadIdx
CONST R1, #1 ; increment
CONST R2, #2 ; N (matrix inner dimension)
CONST R3, #0 ; baseA (matrix A base address)
CONST R4, #4 ; baseB (matrix B base address)
CONST R5, #8 ; baseC (matrix C base address)
DIV R6, R0, R2 ; row = i // N
MUL R7, R6, R2
SUB R7, R0, R7 ; col = i % N
CONST R8, #0 ; acc = 0
CONST R9, #0 ; k = 0
LOOP:
MUL R10, R6, R2
ADD R10, R10, R9
ADD R10, R10, R3 ; addr(A[i]) = row * N + k + baseA
LDR R10, R10 ; load A[i] from global memory
MUL R11, R9, R2
ADD R11, R11, R7
ADD R11, R11, R4 ; addr(B[i]) = k * N + col + baseB
LDR R11, R11 ; load B[i] from global memory
MUL R12, R10, R11
ADD R8, R8, R12 ; acc = acc + A[i] * B[i]
ADD R9, R9, R1 ; increment k
CMP R9, R2
BRn LOOP ; loop while k < N
ADD R9, R5, R0 ; addr(C[i]) = baseC + i
STR R9, R8 ; store C[i] in global memory
RET ; end of kernel
```
# Simulation
tiny-gpu is setup to simulate the execution of both of the above kernels. Before simulating, you'll need to install [iverilog](https://steveicarus.github.io/iverilog/usage/installation.html) and [cocotb](https://docs.cocotb.org/en/stable/install.html):
- Install Verilog compilers with `brew install icarus-verilog` and `pip3 install cocotb`
- Download the latest version of sv2v from https://github.com/zachjs/sv2v/releases, unzip it and put the binary in $PATH.
- Run `mkdir build` in the root directory of this repository.
Once you've installed the pre-requisites, you can run the kernel simulations with `make test_matadd` and `make test_matmul`.
Executing the simulations will output a log file in `test/logs` with the initial data memory state, complete execution trace of the kernel, and final data memory state.
If you look at the initial data memory state logged at the start of the logfile for each, you should see the two start matrices for the calculation, and in the final data memory at the end of the file you should also see the resultant matrix.
Below is a sample of the execution traces, showing on each cycle the execution of every thread within every core, including the current instruction, PC, register values, states, etc.
**For anyone trying to run the simulation or play with this repo, please feel free to DM me on [twitter](https://twitter.com/majmudaradam) if you run into any issues - I want you to get this running!**
# Advanced Functionality
For the sake of simplicity, there were many additional features implemented in modern GPUs that heavily improve performance & functionality that tiny-gpu omits. We'll discuss some of those most critical features in this section.
### Multi-layered Cache & Shared Memory
In modern GPUs, multiple different levels of caches are used to minimize the amount of data that needs to get accessed from global memory. tiny-gpu implements only one cache layer between individual compute units requesting memory and the memory controllers which stores recent cached data.
Implementing multi-layered caches allows frequently accessed data to be cached more locally to where it's being used (with some caches within individual compute cores), minimizing load times for this data.
Different caching algorithms are used to maximize cache-hits - this is a critical dimension that can be improved on to optimize memory access.
Additionally, GPUs often use **shared memory** for threads within the same block to access a single memory space that can be used to share results with other threads.
### Memory Coalescing
Another critical memory optimization used by GPUs is **memory coalescing.** Multiple threads running in parallel often need to access sequential addresses in memory (for example, a group of threads accessing neighboring elements in a matrix) - but each of these memory requests is put in separately.
Memory coalescing is used to analyzing queued memory requests and combine neighboring requests into a single transaction, minimizing time spent on addressing, and making all the requests together.
### Pipelining
In the control flow for tiny-gpu, cores wait for one instruction to be executed on a group of threads before starting execution of the next instruction.
Modern GPUs use **pipelining** to stream execution of multiple sequential instructions at once while ensuring that instructions with dependencies on each other still get executed sequentially.
This helps to maximize resource utilization within cores as resources are not sitting idle while waiting (ex: during async memory requests).
### Warp Scheduling
Another strategy used to maximize resource utilization on course is **warp scheduling.** This approach involves breaking up blocks into individual batches of theads that can be executed together.
Multiple warps can be executed on a single core simultaneously by executing instructions from one warp while another warp is waiting. This is similar to pipelining, but dealing with instructions from different threads.
### Branch Divergence
tiny-gpu assumes that all threads in a single batch end up on the same PC after each instruction, meaning that threads can be executed in parallel for their entire lifetime.
In reality, individual threads could diverge from each other and branch to different lines based on their data. With different PCs, these threads would need to split into separate lines of execution, which requires managing diverging threads & paying attention to when threads converge again.
### Synchronization & Barriers
Another core functionality of modern GPUs is the ability to set **barriers** so that groups of threads in a block can synchronize and wait until all other threads in the same block have gotten to a certain point before continuing execution.
This is useful for cases where threads need to exchange shared data with each other so they can ensure that the data has been fully processed.
# Next Steps
Updates I want to make in the future to improve the design, anyone else is welcome to contribute as well:
- [ ] Add a simple cache for instructions
- [ ] Build an adapter to use GPU with Tiny Tapeout 7
- [ ] Add basic branch divergence
- [ ] Add basic memory coalescing
- [ ] Add basic pipelining
- [ ] Optimize control flow and use of registers to improve cycle time
- [ ] Write a basic graphics kernel or add simple graphics hardware to demonstrate graphics functionality
**For anyone curious to play around or make a contribution, feel free to put up a PR with any improvements you'd like to add 😄**
================================================
FILE: gds/0/gpu.gds
================================================
[File too large to display: 21.3 MB]
================================================
FILE: gds/1/gpu.gds
================================================
[File too large to display: 11.0 MB]
================================================
FILE: src/alu.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// ARITHMETIC-LOGIC UNIT
// > Executes computations on register values
// > In this minimal implementation, the ALU supports the 4 basic arithmetic operations
// > Each thread in each core has it's own ALU
// > ADD, SUB, MUL, DIV instructions are all executed here
module alu (
input wire clk,
input wire reset,
input wire enable, // If current block has less threads then block size, some ALUs will be inactive
input reg [2:0] core_state,
input reg [1:0] decoded_alu_arithmetic_mux,
input reg decoded_alu_output_mux,
input reg [7:0] rs,
input reg [7:0] rt,
output wire [7:0] alu_out
);
localparam ADD = 2'b00,
SUB = 2'b01,
MUL = 2'b10,
DIV = 2'b11;
reg [7:0] alu_out_reg;
assign alu_out = alu_out_reg;
always @(posedge clk) begin
if (reset) begin
alu_out_reg <= 8'b0;
end else if (enable) begin
// Calculate alu_out when core_state = EXECUTE
if (core_state == 3'b101) begin
if (decoded_alu_output_mux == 1) begin
// Set values to compare with NZP register in alu_out[2:0]
alu_out_reg <= {5'b0, (rs - rt > 0), (rs - rt == 0), (rs - rt < 0)};
end else begin
// Execute the specified arithmetic instruction
case (decoded_alu_arithmetic_mux)
ADD: begin
alu_out_reg <= rs + rt;
end
SUB: begin
alu_out_reg <= rs - rt;
end
MUL: begin
alu_out_reg <= rs * rt;
end
DIV: begin
alu_out_reg <= rs / rt;
end
endcase
end
end
end
end
endmodule
================================================
FILE: src/controller.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// MEMORY CONTROLLER
// > Receives memory requests from all cores
// > Throttles requests based on limited external memory bandwidth
// > Waits for responses from external memory and distributes them back to cores
module controller #(
parameter ADDR_BITS = 8,
parameter DATA_BITS = 16,
parameter NUM_CONSUMERS = 4, // The number of consumers accessing memory through this controller
parameter NUM_CHANNELS = 1, // The number of concurrent channels available to send requests to global memory
parameter WRITE_ENABLE = 1 // Whether this memory controller can write to memory (program memory is read-only)
) (
input wire clk,
input wire reset,
// Consumer Interface (Fetchers / LSUs)
input reg [NUM_CONSUMERS-1:0] consumer_read_valid,
input reg [ADDR_BITS-1:0] consumer_read_address [NUM_CONSUMERS-1:0],
output reg [NUM_CONSUMERS-1:0] consumer_read_ready,
output reg [DATA_BITS-1:0] consumer_read_data [NUM_CONSUMERS-1:0],
input reg [NUM_CONSUMERS-1:0] consumer_write_valid,
input reg [ADDR_BITS-1:0] consumer_write_address [NUM_CONSUMERS-1:0],
input reg [DATA_BITS-1:0] consumer_write_data [NUM_CONSUMERS-1:0],
output reg [NUM_CONSUMERS-1:0] consumer_write_ready,
// Memory Interface (Data / Program)
output reg [NUM_CHANNELS-1:0] mem_read_valid,
output reg [ADDR_BITS-1:0] mem_read_address [NUM_CHANNELS-1:0],
input reg [NUM_CHANNELS-1:0] mem_read_ready,
input reg [DATA_BITS-1:0] mem_read_data [NUM_CHANNELS-1:0],
output reg [NUM_CHANNELS-1:0] mem_write_valid,
output reg [ADDR_BITS-1:0] mem_write_address [NUM_CHANNELS-1:0],
output reg [DATA_BITS-1:0] mem_write_data [NUM_CHANNELS-1:0],
input reg [NUM_CHANNELS-1:0] mem_write_ready
);
localparam IDLE = 3'b000,
READ_WAITING = 3'b010,
WRITE_WAITING = 3'b011,
READ_RELAYING = 3'b100,
WRITE_RELAYING = 3'b101;
// Keep track of state for each channel and which jobs each channel is handling
reg [2:0] controller_state [NUM_CHANNELS-1:0];
reg [$clog2(NUM_CONSUMERS)-1:0] current_consumer [NUM_CHANNELS-1:0]; // Which consumer is each channel currently serving
reg [NUM_CONSUMERS-1:0] channel_serving_consumer; // Which channels are being served? Prevents many workers from picking up the same request.
always @(posedge clk) begin
if (reset) begin
mem_read_valid <= 0;
mem_read_address <= 0;
mem_write_valid <= 0;
mem_write_address <= 0;
mem_write_data <= 0;
consumer_read_ready <= 0;
consumer_read_data <= 0;
consumer_write_ready <= 0;
current_consumer <= 0;
controller_state <= 0;
channel_serving_consumer = 0;
end else begin
// For each channel, we handle processing concurrently
for (int i = 0; i < NUM_CHANNELS; i = i + 1) begin
case (controller_state[i])
IDLE: begin
// While this channel is idle, cycle through consumers looking for one with a pending request
for (int j = 0; j < NUM_CONSUMERS; j = j + 1) begin
if (consumer_read_valid[j] && !channel_serving_consumer[j]) begin
channel_serving_consumer[j] = 1;
current_consumer[i] <= j;
mem_read_valid[i] <= 1;
mem_read_address[i] <= consumer_read_address[j];
controller_state[i] <= READ_WAITING;
// Once we find a pending request, pick it up with this channel and stop looking for requests
break;
end else if (consumer_write_valid[j] && !channel_serving_consumer[j]) begin
channel_serving_consumer[j] = 1;
current_consumer[i] <= j;
mem_write_valid[i] <= 1;
mem_write_address[i] <= consumer_write_address[j];
mem_write_data[i] <= consumer_write_data[j];
controller_state[i] <= WRITE_WAITING;
// Once we find a pending request, pick it up with this channel and stop looking for requests
break;
end
end
end
READ_WAITING: begin
// Wait for response from memory for pending read request
if (mem_read_ready[i]) begin
mem_read_valid[i] <= 0;
consumer_read_ready[current_consumer[i]] <= 1;
consumer_read_data[current_consumer[i]] <= mem_read_data[i];
controller_state[i] <= READ_RELAYING;
end
end
WRITE_WAITING: begin
// Wait for response from memory for pending write request
if (mem_write_ready[i]) begin
mem_write_valid[i] <= 0;
consumer_write_ready[current_consumer[i]] <= 1;
controller_state[i] <= WRITE_RELAYING;
end
end
// Wait until consumer acknowledges it received response, then reset
READ_RELAYING: begin
if (!consumer_read_valid[current_consumer[i]]) begin
channel_serving_consumer[current_consumer[i]] = 0;
consumer_read_ready[current_consumer[i]] <= 0;
controller_state[i] <= IDLE;
end
end
WRITE_RELAYING: begin
if (!consumer_write_valid[current_consumer[i]]) begin
channel_serving_consumer[current_consumer[i]] = 0;
consumer_write_ready[current_consumer[i]] <= 0;
controller_state[i] <= IDLE;
end
end
endcase
end
end
end
endmodule
================================================
FILE: src/core.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// COMPUTE CORE
// > Handles processing 1 block at a time
// > The core also has it's own scheduler to manage control flow
// > Each core contains 1 fetcher & decoder, and register files, ALUs, LSUs, PC for each thread
module core #(
parameter DATA_MEM_ADDR_BITS = 8,
parameter DATA_MEM_DATA_BITS = 8,
parameter PROGRAM_MEM_ADDR_BITS = 8,
parameter PROGRAM_MEM_DATA_BITS = 16,
parameter THREADS_PER_BLOCK = 4
) (
input wire clk,
input wire reset,
// Kernel Execution
input wire start,
output wire done,
// Block Metadata
input wire [7:0] block_id,
input wire [$clog2(THREADS_PER_BLOCK):0] thread_count,
// Program Memory
output reg program_mem_read_valid,
output reg [PROGRAM_MEM_ADDR_BITS-1:0] program_mem_read_address,
input reg program_mem_read_ready,
input reg [PROGRAM_MEM_DATA_BITS-1:0] program_mem_read_data,
// Data Memory
output reg [THREADS_PER_BLOCK-1:0] data_mem_read_valid,
output reg [DATA_MEM_ADDR_BITS-1:0] data_mem_read_address [THREADS_PER_BLOCK-1:0],
input reg [THREADS_PER_BLOCK-1:0] data_mem_read_ready,
input reg [DATA_MEM_DATA_BITS-1:0] data_mem_read_data [THREADS_PER_BLOCK-1:0],
output reg [THREADS_PER_BLOCK-1:0] data_mem_write_valid,
output reg [DATA_MEM_ADDR_BITS-1:0] data_mem_write_address [THREADS_PER_BLOCK-1:0],
output reg [DATA_MEM_DATA_BITS-1:0] data_mem_write_data [THREADS_PER_BLOCK-1:0],
input reg [THREADS_PER_BLOCK-1:0] data_mem_write_ready
);
// State
reg [2:0] core_state;
reg [2:0] fetcher_state;
reg [15:0] instruction;
// Intermediate Signals
reg [7:0] current_pc;
wire [7:0] next_pc[THREADS_PER_BLOCK-1:0];
reg [7:0] rs[THREADS_PER_BLOCK-1:0];
reg [7:0] rt[THREADS_PER_BLOCK-1:0];
reg [1:0] lsu_state[THREADS_PER_BLOCK-1:0];
reg [7:0] lsu_out[THREADS_PER_BLOCK-1:0];
wire [7:0] alu_out[THREADS_PER_BLOCK-1:0];
// Decoded Instruction Signals
reg [3:0] decoded_rd_address;
reg [3:0] decoded_rs_address;
reg [3:0] decoded_rt_address;
reg [2:0] decoded_nzp;
reg [7:0] decoded_immediate;
// Decoded Control Signals
reg decoded_reg_write_enable; // Enable writing to a register
reg decoded_mem_read_enable; // Enable reading from memory
reg decoded_mem_write_enable; // Enable writing to memory
reg decoded_nzp_write_enable; // Enable writing to NZP register
reg [1:0] decoded_reg_input_mux; // Select input to register
reg [1:0] decoded_alu_arithmetic_mux; // Select arithmetic operation
reg decoded_alu_output_mux; // Select operation in ALU
reg decoded_pc_mux; // Select source of next PC
reg decoded_ret;
// Fetcher
fetcher #(
.PROGRAM_MEM_ADDR_BITS(PROGRAM_MEM_ADDR_BITS),
.PROGRAM_MEM_DATA_BITS(PROGRAM_MEM_DATA_BITS)
) fetcher_instance (
.clk(clk),
.reset(reset),
.core_state(core_state),
.current_pc(current_pc),
.mem_read_valid(program_mem_read_valid),
.mem_read_address(program_mem_read_address),
.mem_read_ready(program_mem_read_ready),
.mem_read_data(program_mem_read_data),
.fetcher_state(fetcher_state),
.instruction(instruction)
);
// Decoder
decoder decoder_instance (
.clk(clk),
.reset(reset),
.core_state(core_state),
.instruction(instruction),
.decoded_rd_address(decoded_rd_address),
.decoded_rs_address(decoded_rs_address),
.decoded_rt_address(decoded_rt_address),
.decoded_nzp(decoded_nzp),
.decoded_immediate(decoded_immediate),
.decoded_reg_write_enable(decoded_reg_write_enable),
.decoded_mem_read_enable(decoded_mem_read_enable),
.decoded_mem_write_enable(decoded_mem_write_enable),
.decoded_nzp_write_enable(decoded_nzp_write_enable),
.decoded_reg_input_mux(decoded_reg_input_mux),
.decoded_alu_arithmetic_mux(decoded_alu_arithmetic_mux),
.decoded_alu_output_mux(decoded_alu_output_mux),
.decoded_pc_mux(decoded_pc_mux),
.decoded_ret(decoded_ret)
);
// Scheduler
scheduler #(
.THREADS_PER_BLOCK(THREADS_PER_BLOCK),
) scheduler_instance (
.clk(clk),
.reset(reset),
.start(start),
.fetcher_state(fetcher_state),
.core_state(core_state),
.decoded_mem_read_enable(decoded_mem_read_enable),
.decoded_mem_write_enable(decoded_mem_write_enable),
.decoded_ret(decoded_ret),
.lsu_state(lsu_state),
.current_pc(current_pc),
.next_pc(next_pc),
.done(done)
);
// Dedicated ALU, LSU, registers, & PC unit for each thread this core has capacity for
genvar i;
generate
for (i = 0; i < THREADS_PER_BLOCK; i = i + 1) begin : threads
// ALU
alu alu_instance (
.clk(clk),
.reset(reset),
.enable(i < thread_count),
.core_state(core_state),
.decoded_alu_arithmetic_mux(decoded_alu_arithmetic_mux),
.decoded_alu_output_mux(decoded_alu_output_mux),
.rs(rs[i]),
.rt(rt[i]),
.alu_out(alu_out[i])
);
// LSU
lsu lsu_instance (
.clk(clk),
.reset(reset),
.enable(i < thread_count),
.core_state(core_state),
.decoded_mem_read_enable(decoded_mem_read_enable),
.decoded_mem_write_enable(decoded_mem_write_enable),
.mem_read_valid(data_mem_read_valid[i]),
.mem_read_address(data_mem_read_address[i]),
.mem_read_ready(data_mem_read_ready[i]),
.mem_read_data(data_mem_read_data[i]),
.mem_write_valid(data_mem_write_valid[i]),
.mem_write_address(data_mem_write_address[i]),
.mem_write_data(data_mem_write_data[i]),
.mem_write_ready(data_mem_write_ready[i]),
.rs(rs[i]),
.rt(rt[i]),
.lsu_state(lsu_state[i]),
.lsu_out(lsu_out[i])
);
// Register File
registers #(
.THREADS_PER_BLOCK(THREADS_PER_BLOCK),
.THREAD_ID(i),
.DATA_BITS(DATA_MEM_DATA_BITS),
) register_instance (
.clk(clk),
.reset(reset),
.enable(i < thread_count),
.block_id(block_id),
.core_state(core_state),
.decoded_reg_write_enable(decoded_reg_write_enable),
.decoded_reg_input_mux(decoded_reg_input_mux),
.decoded_rd_address(decoded_rd_address),
.decoded_rs_address(decoded_rs_address),
.decoded_rt_address(decoded_rt_address),
.decoded_immediate(decoded_immediate),
.alu_out(alu_out[i]),
.lsu_out(lsu_out[i]),
.rs(rs[i]),
.rt(rt[i])
);
// Program Counter
pc #(
.DATA_MEM_DATA_BITS(DATA_MEM_DATA_BITS),
.PROGRAM_MEM_ADDR_BITS(PROGRAM_MEM_ADDR_BITS)
) pc_instance (
.clk(clk),
.reset(reset),
.enable(i < thread_count),
.core_state(core_state),
.decoded_nzp(decoded_nzp),
.decoded_immediate(decoded_immediate),
.decoded_nzp_write_enable(decoded_nzp_write_enable),
.decoded_pc_mux(decoded_pc_mux),
.alu_out(alu_out[i]),
.current_pc(current_pc),
.next_pc(next_pc[i])
);
end
endgenerate
endmodule
================================================
FILE: src/dcr.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// DEVICE CONTROL REGISTER
// > Used to configure high-level settings
// > In this minimal example, the DCR is used to configure the number of threads to run for the kernel
module dcr (
input wire clk,
input wire reset,
input wire device_control_write_enable,
input wire [7:0] device_control_data,
output wire [7:0] thread_count,
);
// Store device control data in dedicated register
reg [7:0] device_conrol_register;
assign thread_count = device_conrol_register[7:0];
always @(posedge clk) begin
if (reset) begin
device_conrol_register <= 8'b0;
end else begin
if (device_control_write_enable) begin
device_conrol_register <= device_control_data;
end
end
end
endmodule
================================================
FILE: src/decoder.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// INSTRUCTION DECODER
// > Decodes an instruction into the control signals necessary to execute it
// > Each core has it's own decoder
module decoder (
input wire clk,
input wire reset,
input reg [2:0] core_state,
input reg [15:0] instruction,
// Instruction Signals
output reg [3:0] decoded_rd_address,
output reg [3:0] decoded_rs_address,
output reg [3:0] decoded_rt_address,
output reg [2:0] decoded_nzp,
output reg [7:0] decoded_immediate,
// Control Signals
output reg decoded_reg_write_enable, // Enable writing to a register
output reg decoded_mem_read_enable, // Enable reading from memory
output reg decoded_mem_write_enable, // Enable writing to memory
output reg decoded_nzp_write_enable, // Enable writing to NZP register
output reg [1:0] decoded_reg_input_mux, // Select input to register
output reg [1:0] decoded_alu_arithmetic_mux, // Select arithmetic operation
output reg decoded_alu_output_mux, // Select operation in ALU
output reg decoded_pc_mux, // Select source of next PC
// Return (finished executing thread)
output reg decoded_ret
);
localparam NOP = 4'b0000,
BRnzp = 4'b0001,
CMP = 4'b0010,
ADD = 4'b0011,
SUB = 4'b0100,
MUL = 4'b0101,
DIV = 4'b0110,
LDR = 4'b0111,
STR = 4'b1000,
CONST = 4'b1001,
RET = 4'b1111;
always @(posedge clk) begin
if (reset) begin
decoded_rd_address <= 0;
decoded_rs_address <= 0;
decoded_rt_address <= 0;
decoded_immediate <= 0;
decoded_nzp <= 0;
decoded_reg_write_enable <= 0;
decoded_mem_read_enable <= 0;
decoded_mem_write_enable <= 0;
decoded_nzp_write_enable <= 0;
decoded_reg_input_mux <= 0;
decoded_alu_arithmetic_mux <= 0;
decoded_alu_output_mux <= 0;
decoded_pc_mux <= 0;
decoded_ret <= 0;
end else begin
// Decode when core_state = DECODE
if (core_state == 3'b010) begin
// Get instruction signals from instruction every time
decoded_rd_address <= instruction[11:8];
decoded_rs_address <= instruction[7:4];
decoded_rt_address <= instruction[3:0];
decoded_immediate <= instruction[7:0];
decoded_nzp <= instruction[11:9];
// Control signals reset on every decode and set conditionally by instruction
decoded_reg_write_enable <= 0;
decoded_mem_read_enable <= 0;
decoded_mem_write_enable <= 0;
decoded_nzp_write_enable <= 0;
decoded_reg_input_mux <= 0;
decoded_alu_arithmetic_mux <= 0;
decoded_alu_output_mux <= 0;
decoded_pc_mux <= 0;
decoded_ret <= 0;
// Set the control signals for each instruction
case (instruction[15:12])
NOP: begin
// no-op
end
BRnzp: begin
decoded_pc_mux <= 1;
end
CMP: begin
decoded_alu_output_mux <= 1;
decoded_nzp_write_enable <= 1;
end
ADD: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b00;
decoded_alu_arithmetic_mux <= 2'b00;
end
SUB: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b00;
decoded_alu_arithmetic_mux <= 2'b01;
end
MUL: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b00;
decoded_alu_arithmetic_mux <= 2'b10;
end
DIV: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b00;
decoded_alu_arithmetic_mux <= 2'b11;
end
LDR: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b01;
decoded_mem_read_enable <= 1;
end
STR: begin
decoded_mem_write_enable <= 1;
end
CONST: begin
decoded_reg_write_enable <= 1;
decoded_reg_input_mux <= 2'b10;
end
RET: begin
decoded_ret <= 1;
end
endcase
end
end
end
endmodule
================================================
FILE: src/dispatch.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// BLOCK DISPATCH
// > The GPU has one dispatch unit at the top level
// > Manages processing of threads and marks kernel execution as done
// > Sends off batches of threads in blocks to be executed by available compute cores
module dispatch #(
parameter NUM_CORES = 2,
parameter THREADS_PER_BLOCK = 4
) (
input wire clk,
input wire reset,
input wire start,
// Kernel Metadata
input wire [7:0] thread_count,
// Core States
input reg [NUM_CORES-1:0] core_done,
output reg [NUM_CORES-1:0] core_start,
output reg [NUM_CORES-1:0] core_reset,
output reg [7:0] core_block_id [NUM_CORES-1:0],
output reg [$clog2(THREADS_PER_BLOCK):0] core_thread_count [NUM_CORES-1:0],
// Kernel Execution
output reg done
);
// Calculate the total number of blocks based on total threads & threads per block
wire [7:0] total_blocks;
assign total_blocks = (thread_count + THREADS_PER_BLOCK - 1) / THREADS_PER_BLOCK;
// Keep track of how many blocks have been processed
reg [7:0] blocks_dispatched; // How many blocks have been sent to cores?
reg [7:0] blocks_done; // How many blocks have finished processing?
reg start_execution; // EDA: Unimportant hack used because of EDA tooling
always @(posedge clk) begin
if (reset) begin
done <= 0;
blocks_dispatched = 0;
blocks_done = 0;
start_execution <= 0;
for (int i = 0; i < NUM_CORES; i++) begin
core_start[i] <= 0;
core_reset[i] <= 1;
core_block_id[i] <= 0;
core_thread_count[i] <= THREADS_PER_BLOCK;
end
end else if (start) begin
// EDA: Indirect way to get @(posedge start) without driving from 2 different clocks
if (!start_execution) begin
start_execution <= 1;
for (int i = 0; i < NUM_CORES; i++) begin
core_reset[i] <= 1;
end
end
// If the last block has finished processing, mark this kernel as done executing
if (blocks_done == total_blocks) begin
done <= 1;
end
for (int i = 0; i < NUM_CORES; i++) begin
if (core_reset[i]) begin
core_reset[i] <= 0;
// If this core was just reset, check if there are more blocks to be dispatched
if (blocks_dispatched < total_blocks) begin
core_start[i] <= 1;
core_block_id[i] <= blocks_dispatched;
core_thread_count[i] <= (blocks_dispatched == total_blocks - 1)
? thread_count - (blocks_dispatched * THREADS_PER_BLOCK)
: THREADS_PER_BLOCK;
blocks_dispatched = blocks_dispatched + 1;
end
end
end
for (int i = 0; i < NUM_CORES; i++) begin
if (core_start[i] && core_done[i]) begin
// If a core just finished executing it's current block, reset it
core_reset[i] <= 1;
core_start[i] <= 0;
blocks_done = blocks_done + 1;
end
end
end
end
endmodule
================================================
FILE: src/fetcher.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// INSTRUCTION FETCHER
// > Retrieves the instruction at the current PC from global data memory
// > Each core has it's own fetcher
module fetcher #(
parameter PROGRAM_MEM_ADDR_BITS = 8,
parameter PROGRAM_MEM_DATA_BITS = 16
) (
input wire clk,
input wire reset,
// Execution State
input reg [2:0] core_state,
input reg [7:0] current_pc,
// Program Memory
output reg mem_read_valid,
output reg [PROGRAM_MEM_ADDR_BITS-1:0] mem_read_address,
input reg mem_read_ready,
input reg [PROGRAM_MEM_DATA_BITS-1:0] mem_read_data,
// Fetcher Output
output reg [2:0] fetcher_state,
output reg [PROGRAM_MEM_DATA_BITS-1:0] instruction,
);
localparam IDLE = 3'b000,
FETCHING = 3'b001,
FETCHED = 3'b010;
always @(posedge clk) begin
if (reset) begin
fetcher_state <= IDLE;
mem_read_valid <= 0;
mem_read_address <= 0;
instruction <= {PROGRAM_MEM_DATA_BITS{1'b0}};
end else begin
case (fetcher_state)
IDLE: begin
// Start fetching when core_state = FETCH
if (core_state == 3'b001) begin
fetcher_state <= FETCHING;
mem_read_valid <= 1;
mem_read_address <= current_pc;
end
end
FETCHING: begin
// Wait for response from program memory
if (mem_read_ready) begin
fetcher_state <= FETCHED;
instruction <= mem_read_data; // Store the instruction when received
mem_read_valid <= 0;
end
end
FETCHED: begin
// Reset when core_state = DECODE
if (core_state == 3'b010) begin
fetcher_state <= IDLE;
end
end
endcase
end
end
endmodule
================================================
FILE: src/gpu.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// GPU
// > Built to use an external async memory with multi-channel read/write
// > Assumes that the program is loaded into program memory, data into data memory, and threads into
// the device control register before the start signal is triggered
// > Has memory controllers to interface between external memory and its multiple cores
// > Configurable number of cores and thread capacity per core
module gpu #(
parameter DATA_MEM_ADDR_BITS = 8, // Number of bits in data memory address (256 rows)
parameter DATA_MEM_DATA_BITS = 8, // Number of bits in data memory value (8 bit data)
parameter DATA_MEM_NUM_CHANNELS = 4, // Number of concurrent channels for sending requests to data memory
parameter PROGRAM_MEM_ADDR_BITS = 8, // Number of bits in program memory address (256 rows)
parameter PROGRAM_MEM_DATA_BITS = 16, // Number of bits in program memory value (16 bit instruction)
parameter PROGRAM_MEM_NUM_CHANNELS = 1, // Number of concurrent channels for sending requests to program memory
parameter NUM_CORES = 2, // Number of cores to include in this GPU
parameter THREADS_PER_BLOCK = 4 // Number of threads to handle per block (determines the compute resources of each core)
) (
input wire clk,
input wire reset,
// Kernel Execution
input wire start,
output wire done,
// Device Control Register
input wire device_control_write_enable,
input wire [7:0] device_control_data,
// Program Memory
output wire [PROGRAM_MEM_NUM_CHANNELS-1:0] program_mem_read_valid,
output wire [PROGRAM_MEM_ADDR_BITS-1:0] program_mem_read_address [PROGRAM_MEM_NUM_CHANNELS-1:0],
input wire [PROGRAM_MEM_NUM_CHANNELS-1:0] program_mem_read_ready,
input wire [PROGRAM_MEM_DATA_BITS-1:0] program_mem_read_data [PROGRAM_MEM_NUM_CHANNELS-1:0],
// Data Memory
output wire [DATA_MEM_NUM_CHANNELS-1:0] data_mem_read_valid,
output wire [DATA_MEM_ADDR_BITS-1:0] data_mem_read_address [DATA_MEM_NUM_CHANNELS-1:0],
input wire [DATA_MEM_NUM_CHANNELS-1:0] data_mem_read_ready,
input wire [DATA_MEM_DATA_BITS-1:0] data_mem_read_data [DATA_MEM_NUM_CHANNELS-1:0],
output wire [DATA_MEM_NUM_CHANNELS-1:0] data_mem_write_valid,
output wire [DATA_MEM_ADDR_BITS-1:0] data_mem_write_address [DATA_MEM_NUM_CHANNELS-1:0],
output wire [DATA_MEM_DATA_BITS-1:0] data_mem_write_data [DATA_MEM_NUM_CHANNELS-1:0],
input wire [DATA_MEM_NUM_CHANNELS-1:0] data_mem_write_ready
);
// Control
wire [7:0] thread_count;
// Compute Core State
reg [NUM_CORES-1:0] core_start;
reg [NUM_CORES-1:0] core_reset;
reg [NUM_CORES-1:0] core_done;
reg [7:0] core_block_id [NUM_CORES-1:0];
reg [$clog2(THREADS_PER_BLOCK):0] core_thread_count [NUM_CORES-1:0];
// LSU <> Data Memory Controller Channels
localparam NUM_LSUS = NUM_CORES * THREADS_PER_BLOCK;
reg [NUM_LSUS-1:0] lsu_read_valid;
reg [DATA_MEM_ADDR_BITS-1:0] lsu_read_address [NUM_LSUS-1:0];
reg [NUM_LSUS-1:0] lsu_read_ready;
reg [DATA_MEM_DATA_BITS-1:0] lsu_read_data [NUM_LSUS-1:0];
reg [NUM_LSUS-1:0] lsu_write_valid;
reg [DATA_MEM_ADDR_BITS-1:0] lsu_write_address [NUM_LSUS-1:0];
reg [DATA_MEM_DATA_BITS-1:0] lsu_write_data [NUM_LSUS-1:0];
reg [NUM_LSUS-1:0] lsu_write_ready;
// Fetcher <> Program Memory Controller Channels
localparam NUM_FETCHERS = NUM_CORES;
reg [NUM_FETCHERS-1:0] fetcher_read_valid;
reg [PROGRAM_MEM_ADDR_BITS-1:0] fetcher_read_address [NUM_FETCHERS-1:0];
reg [NUM_FETCHERS-1:0] fetcher_read_ready;
reg [PROGRAM_MEM_DATA_BITS-1:0] fetcher_read_data [NUM_FETCHERS-1:0];
// Device Control Register
dcr dcr_instance (
.clk(clk),
.reset(reset),
.device_control_write_enable(device_control_write_enable),
.device_control_data(device_control_data),
.thread_count(thread_count)
);
// Data Memory Controller
controller #(
.ADDR_BITS(DATA_MEM_ADDR_BITS),
.DATA_BITS(DATA_MEM_DATA_BITS),
.NUM_CONSUMERS(NUM_LSUS),
.NUM_CHANNELS(DATA_MEM_NUM_CHANNELS)
) data_memory_controller (
.clk(clk),
.reset(reset),
.consumer_read_valid(lsu_read_valid),
.consumer_read_address(lsu_read_address),
.consumer_read_ready(lsu_read_ready),
.consumer_read_data(lsu_read_data),
.consumer_write_valid(lsu_write_valid),
.consumer_write_address(lsu_write_address),
.consumer_write_data(lsu_write_data),
.consumer_write_ready(lsu_write_ready),
.mem_read_valid(data_mem_read_valid),
.mem_read_address(data_mem_read_address),
.mem_read_ready(data_mem_read_ready),
.mem_read_data(data_mem_read_data),
.mem_write_valid(data_mem_write_valid),
.mem_write_address(data_mem_write_address),
.mem_write_data(data_mem_write_data),
.mem_write_ready(data_mem_write_ready)
);
// Program Memory Controller
controller #(
.ADDR_BITS(PROGRAM_MEM_ADDR_BITS),
.DATA_BITS(PROGRAM_MEM_DATA_BITS),
.NUM_CONSUMERS(NUM_FETCHERS),
.NUM_CHANNELS(PROGRAM_MEM_NUM_CHANNELS),
.WRITE_ENABLE(0)
) program_memory_controller (
.clk(clk),
.reset(reset),
.consumer_read_valid(fetcher_read_valid),
.consumer_read_address(fetcher_read_address),
.consumer_read_ready(fetcher_read_ready),
.consumer_read_data(fetcher_read_data),
.mem_read_valid(program_mem_read_valid),
.mem_read_address(program_mem_read_address),
.mem_read_ready(program_mem_read_ready),
.mem_read_data(program_mem_read_data),
);
// Dispatcher
dispatch #(
.NUM_CORES(NUM_CORES),
.THREADS_PER_BLOCK(THREADS_PER_BLOCK)
) dispatch_instance (
.clk(clk),
.reset(reset),
.start(start),
.thread_count(thread_count),
.core_done(core_done),
.core_start(core_start),
.core_reset(core_reset),
.core_block_id(core_block_id),
.core_thread_count(core_thread_count),
.done(done)
);
// Compute Cores
genvar i;
generate
for (i = 0; i < NUM_CORES; i = i + 1) begin : cores
// EDA: We create separate signals here to pass to cores because of a requirement
// by the OpenLane EDA flow (uses Verilog 2005) that prevents slicing the top-level signals
reg [THREADS_PER_BLOCK-1:0] core_lsu_read_valid;
reg [DATA_MEM_ADDR_BITS-1:0] core_lsu_read_address [THREADS_PER_BLOCK-1:0];
reg [THREADS_PER_BLOCK-1:0] core_lsu_read_ready;
reg [DATA_MEM_DATA_BITS-1:0] core_lsu_read_data [THREADS_PER_BLOCK-1:0];
reg [THREADS_PER_BLOCK-1:0] core_lsu_write_valid;
reg [DATA_MEM_ADDR_BITS-1:0] core_lsu_write_address [THREADS_PER_BLOCK-1:0];
reg [DATA_MEM_DATA_BITS-1:0] core_lsu_write_data [THREADS_PER_BLOCK-1:0];
reg [THREADS_PER_BLOCK-1:0] core_lsu_write_ready;
// Pass through signals between LSUs and data memory controller
genvar j;
for (j = 0; j < THREADS_PER_BLOCK; j = j + 1) begin
localparam lsu_index = i * THREADS_PER_BLOCK + j;
always @(posedge clk) begin
lsu_read_valid[lsu_index] <= core_lsu_read_valid[j];
lsu_read_address[lsu_index] <= core_lsu_read_address[j];
lsu_write_valid[lsu_index] <= core_lsu_write_valid[j];
lsu_write_address[lsu_index] <= core_lsu_write_address[j];
lsu_write_data[lsu_index] <= core_lsu_write_data[j];
core_lsu_read_ready[j] <= lsu_read_ready[lsu_index];
core_lsu_read_data[j] <= lsu_read_data[lsu_index];
core_lsu_write_ready[j] <= lsu_write_ready[lsu_index];
end
end
// Compute Core
core #(
.DATA_MEM_ADDR_BITS(DATA_MEM_ADDR_BITS),
.DATA_MEM_DATA_BITS(DATA_MEM_DATA_BITS),
.PROGRAM_MEM_ADDR_BITS(PROGRAM_MEM_ADDR_BITS),
.PROGRAM_MEM_DATA_BITS(PROGRAM_MEM_DATA_BITS),
.THREADS_PER_BLOCK(THREADS_PER_BLOCK),
) core_instance (
.clk(clk),
.reset(core_reset[i]),
.start(core_start[i]),
.done(core_done[i]),
.block_id(core_block_id[i]),
.thread_count(core_thread_count[i]),
.program_mem_read_valid(fetcher_read_valid[i]),
.program_mem_read_address(fetcher_read_address[i]),
.program_mem_read_ready(fetcher_read_ready[i]),
.program_mem_read_data(fetcher_read_data[i]),
.data_mem_read_valid(core_lsu_read_valid),
.data_mem_read_address(core_lsu_read_address),
.data_mem_read_ready(core_lsu_read_ready),
.data_mem_read_data(core_lsu_read_data),
.data_mem_write_valid(core_lsu_write_valid),
.data_mem_write_address(core_lsu_write_address),
.data_mem_write_data(core_lsu_write_data),
.data_mem_write_ready(core_lsu_write_ready)
);
end
endgenerate
endmodule
================================================
FILE: src/lsu.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// LOAD-STORE UNIT
// > Handles asynchronous memory load and store operations and waits for response
// > Each thread in each core has it's own LSU
// > LDR, STR instructions are executed here
module lsu (
input wire clk,
input wire reset,
input wire enable, // If current block has less threads then block size, some LSUs will be inactive
// State
input reg [2:0] core_state,
// Memory Control Sgiansl
input reg decoded_mem_read_enable,
input reg decoded_mem_write_enable,
// Registers
input reg [7:0] rs,
input reg [7:0] rt,
// Data Memory
output reg mem_read_valid,
output reg [7:0] mem_read_address,
input reg mem_read_ready,
input reg [7:0] mem_read_data,
output reg mem_write_valid,
output reg [7:0] mem_write_address,
output reg [7:0] mem_write_data,
input reg mem_write_ready,
// LSU Outputs
output reg [1:0] lsu_state,
output reg [7:0] lsu_out
);
localparam IDLE = 2'b00, REQUESTING = 2'b01, WAITING = 2'b10, DONE = 2'b11;
always @(posedge clk) begin
if (reset) begin
lsu_state <= IDLE;
lsu_out <= 0;
mem_read_valid <= 0;
mem_read_address <= 0;
mem_write_valid <= 0;
mem_write_address <= 0;
mem_write_data <= 0;
end else if (enable) begin
// If memory read enable is triggered (LDR instruction)
if (decoded_mem_read_enable) begin
case (lsu_state)
IDLE: begin
// Only read when core_state = REQUEST
if (core_state == 3'b011) begin
lsu_state <= REQUESTING;
end
end
REQUESTING: begin
mem_read_valid <= 1;
mem_read_address <= rs;
lsu_state <= WAITING;
end
WAITING: begin
if (mem_read_ready == 1) begin
mem_read_valid <= 0;
lsu_out <= mem_read_data;
lsu_state <= DONE;
end
end
DONE: begin
// Reset when core_state = UPDATE
if (core_state == 3'b110) begin
lsu_state <= IDLE;
end
end
endcase
end
// If memory write enable is triggered (STR instruction)
if (decoded_mem_write_enable) begin
case (lsu_state)
IDLE: begin
// Only read when core_state = REQUEST
if (core_state == 3'b011) begin
lsu_state <= REQUESTING;
end
end
REQUESTING: begin
mem_write_valid <= 1;
mem_write_address <= rs;
mem_write_data <= rt;
lsu_state <= WAITING;
end
WAITING: begin
if (mem_write_ready) begin
mem_write_valid <= 0;
lsu_state <= DONE;
end
end
DONE: begin
// Reset when core_state = UPDATE
if (core_state == 3'b110) begin
lsu_state <= IDLE;
end
end
endcase
end
end
end
endmodule
================================================
FILE: src/pc.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// PROGRAM COUNTER
// > Calculates the next PC for each thread to update to (but currently we assume all threads
// update to the same PC and don't support branch divergence)
// > Currently, each thread in each core has it's own calculation for next PC
// > The NZP register value is set by the CMP instruction (based on >/=/< comparison) to
// initiate the BRnzp instruction for branching
module pc #(
parameter DATA_MEM_DATA_BITS = 8,
parameter PROGRAM_MEM_ADDR_BITS = 8
) (
input wire clk,
input wire reset,
input wire enable, // If current block has less threads then block size, some PCs will be inactive
// State
input reg [2:0] core_state,
// Control Signals
input reg [2:0] decoded_nzp,
input reg [DATA_MEM_DATA_BITS-1:0] decoded_immediate,
input reg decoded_nzp_write_enable,
input reg decoded_pc_mux,
// ALU Output - used for alu_out[2:0] to compare with NZP register
input reg [DATA_MEM_DATA_BITS-1:0] alu_out,
// Current & Next PCs
input reg [PROGRAM_MEM_ADDR_BITS-1:0] current_pc,
output reg [PROGRAM_MEM_ADDR_BITS-1:0] next_pc
);
reg [2:0] nzp;
always @(posedge clk) begin
if (reset) begin
nzp <= 3'b0;
next_pc <= 0;
end else if (enable) begin
// Update PC when core_state = EXECUTE
if (core_state == 3'b101) begin
if (decoded_pc_mux == 1) begin
if (((nzp & decoded_nzp) != 3'b0)) begin
// On BRnzp instruction, branch to immediate if NZP case matches previous CMP
next_pc <= decoded_immediate;
end else begin
// Otherwise, just update to PC + 1 (next line)
next_pc <= current_pc + 1;
end
end else begin
// By default update to PC + 1 (next line)
next_pc <= current_pc + 1;
end
end
// Store NZP when core_state = UPDATE
if (core_state == 3'b110) begin
// Write to NZP register on CMP instruction
if (decoded_nzp_write_enable) begin
nzp[2] <= alu_out[2];
nzp[1] <= alu_out[1];
nzp[0] <= alu_out[0];
end
end
end
end
endmodule
================================================
FILE: src/registers.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// REGISTER FILE
// > Each thread within each core has it's own register file with 13 free registers and 3 read-only registers
// > Read-only registers hold the familiar %blockIdx, %blockDim, and %threadIdx values critical to SIMD
module registers #(
parameter THREADS_PER_BLOCK = 4,
parameter THREAD_ID = 0,
parameter DATA_BITS = 8
) (
input wire clk,
input wire reset,
input wire enable, // If current block has less threads then block size, some registers will be inactive
// Kernel Execution
input reg [7:0] block_id,
// State
input reg [2:0] core_state,
// Instruction Signals
input reg [3:0] decoded_rd_address,
input reg [3:0] decoded_rs_address,
input reg [3:0] decoded_rt_address,
// Control Signals
input reg decoded_reg_write_enable,
input reg [1:0] decoded_reg_input_mux,
input reg [DATA_BITS-1:0] decoded_immediate,
// Thread Unit Outputs
input reg [DATA_BITS-1:0] alu_out,
input reg [DATA_BITS-1:0] lsu_out,
// Registers
output reg [7:0] rs,
output reg [7:0] rt
);
localparam ARITHMETIC = 2'b00,
MEMORY = 2'b01,
CONSTANT = 2'b10;
// 16 registers per thread (13 free registers and 3 read-only registers)
reg [7:0] registers[15:0];
always @(posedge clk) begin
if (reset) begin
// Empty rs, rt
rs <= 0;
rt <= 0;
// Initialize all free registers
registers[0] <= 8'b0;
registers[1] <= 8'b0;
registers[2] <= 8'b0;
registers[3] <= 8'b0;
registers[4] <= 8'b0;
registers[5] <= 8'b0;
registers[6] <= 8'b0;
registers[7] <= 8'b0;
registers[8] <= 8'b0;
registers[9] <= 8'b0;
registers[10] <= 8'b0;
registers[11] <= 8'b0;
registers[12] <= 8'b0;
// Initialize read-only registers
registers[13] <= 8'b0; // %blockIdx
registers[14] <= THREADS_PER_BLOCK; // %blockDim
registers[15] <= THREAD_ID; // %threadIdx
end else if (enable) begin
// [Bad Solution] Shouldn't need to set this every cycle
registers[13] <= block_id; // Update the block_id when a new block is issued from dispatcher
// Fill rs/rt when core_state = REQUEST
if (core_state == 3'b011) begin
rs <= registers[decoded_rs_address];
rt <= registers[decoded_rt_address];
end
// Store rd when core_state = UPDATE
if (core_state == 3'b110) begin
// Only allow writing to R0 - R12
if (decoded_reg_write_enable && decoded_rd_address < 13) begin
case (decoded_reg_input_mux)
ARITHMETIC: begin
// ADD, SUB, MUL, DIV
registers[decoded_rd_address] <= alu_out;
end
MEMORY: begin
// LDR
registers[decoded_rd_address] <= lsu_out;
end
CONSTANT: begin
// CONST
registers[decoded_rd_address] <= decoded_immediate;
end
endcase
end
end
end
end
endmodule
================================================
FILE: src/scheduler.sv
================================================
`default_nettype none
`timescale 1ns/1ns
// SCHEDULER
// > Manages the entire control flow of a single compute core processing 1 block
// 1. FETCH - Retrieve instruction at current program counter (PC) from program memory
// 2. DECODE - Decode the instruction into the relevant control signals
// 3. REQUEST - If we have an instruction that accesses memory, trigger the async memory requests from LSUs
// 4. WAIT - Wait for all async memory requests to resolve (if applicable)
// 5. EXECUTE - Execute computations on retrieved data from registers / memory
// 6. UPDATE - Update register values (including NZP register) and program counter
// > Each core has it's own scheduler where multiple threads can be processed with
// the same control flow at once.
// > Technically, different instructions can branch to different PCs, requiring "branch divergence." In
// this minimal implementation, we assume no branch divergence (naive approach for simplicity)
module scheduler #(
parameter THREADS_PER_BLOCK = 4,
) (
input wire clk,
input wire reset,
input wire start,
// Control Signals
input reg decoded_mem_read_enable,
input reg decoded_mem_write_enable,
input reg decoded_ret,
// Memory Access State
input reg [2:0] fetcher_state,
input reg [1:0] lsu_state [THREADS_PER_BLOCK-1:0],
// Current & Next PC
output reg [7:0] current_pc,
input reg [7:0] next_pc [THREADS_PER_BLOCK-1:0],
// Execution State
output reg [2:0] core_state,
output reg done
);
localparam IDLE = 3'b000, // Waiting to start
FETCH = 3'b001, // Fetch instructions from program memory
DECODE = 3'b010, // Decode instructions into control signals
REQUEST = 3'b011, // Request data from registers or memory
WAIT = 3'b100, // Wait for response from memory if necessary
EXECUTE = 3'b101, // Execute ALU and PC calculations
UPDATE = 3'b110, // Update registers, NZP, and PC
DONE = 3'b111; // Done executing this block
always @(posedge clk) begin
if (reset) begin
current_pc <= 0;
core_state <= IDLE;
done <= 0;
end else begin
case (core_state)
IDLE: begin
// Here after reset (before kernel is launched, or after previous block has been processed)
if (start) begin
// Start by fetching the next instruction for this block based on PC
core_state <= FETCH;
end
end
FETCH: begin
// Move on once fetcher_state = FETCHED
if (fetcher_state == 3'b010) begin
core_state <= DECODE;
end
end
DECODE: begin
// Decode is synchronous so we move on after one cycle
core_state <= REQUEST;
end
REQUEST: begin
// Request is synchronous so we move on after one cycle
core_state <= WAIT;
end
WAIT: begin
// Wait for all LSUs to finish their request before continuing
reg any_lsu_waiting = 1'b0;
for (int i = 0; i < THREADS_PER_BLOCK; i++) begin
// Make sure no lsu_state = REQUESTING or WAITING
if (lsu_state[i] == 2'b01 || lsu_state[i] == 2'b10) begin
any_lsu_waiting = 1'b1;
break;
end
end
// If no LSU is waiting for a response, move onto the next stage
if (!any_lsu_waiting) begin
core_state <= EXECUTE;
end
end
EXECUTE: begin
// Execute is synchronous so we move on after one cycle
core_state <= UPDATE;
end
UPDATE: begin
if (decoded_ret) begin
// If we reach a RET instruction, this block is done executing
done <= 1;
core_state <= DONE;
end else begin
// TODO: Branch divergence. For now assume all next_pc converge
current_pc <= next_pc[THREADS_PER_BLOCK-1];
// Update is synchronous so we move on after one cycle
core_state <= FETCH;
end
end
DONE: begin
// no-op
end
endcase
end
end
endmodule
================================================
FILE: test/__init__.py
================================================
================================================
FILE: test/helpers/format.py
================================================
from typing import List, Optional
from .logger import logger
def format_register(register: int) -> str:
if register < 13:
return f"R{register}"
if register == 13:
return f"%blockIdx"
if register == 14:
return f"%blockDim"
if register == 15:
return f"%threadIdx"
def format_instruction(instruction: str) -> str:
opcode = instruction[0:4]
rd = format_register(int(instruction[4:8], 2))
rs = format_register(int(instruction[8:12], 2))
rt = format_register(int(instruction[12:16], 2))
n = "N" if instruction[4] == 1 else ""
z = "Z" if instruction[5] == 1 else ""
p = "P" if instruction[6] == 1 else ""
imm = f"#{int(instruction[8:16], 2)}"
if opcode == "0000":
return "NOP"
elif opcode == "0001":
return f"BRnzp {n}{z}{p}, {imm}"
elif opcode == "0010":
return f"CMP {rs}, {rt}"
elif opcode == "0011":
return f"ADD {rd}, {rs}, {rt}"
elif opcode == "0100":
return f"SUB {rd}, {rs}, {rt}"
elif opcode == "0101":
return f"MUL {rd}, {rs}, {rt}"
elif opcode == "0110":
return f"DIV {rd}, {rs}, {rt}"
elif opcode == "0111":
return f"LDR {rd}, {rs}"
elif opcode == "1000":
return f"STR {rs}, {rt}"
elif opcode == "1001":
return f"CONST {rd}, {imm}"
elif opcode == "1111":
return "RET"
return "UNKNOWN"
def format_core_state(core_state: str) -> str:
core_state_map = {
"000": "IDLE",
"001": "FETCH",
"010": "DECODE",
"011": "REQUEST",
"100": "WAIT",
"101": "EXECUTE",
"110": "UPDATE",
"111": "DONE"
}
return core_state_map[core_state]
def format_fetcher_state(fetcher_state: str) -> str:
fetcher_state_map = {
"000": "IDLE",
"001": "FETCHING",
"010": "FETCHED"
}
return fetcher_state_map[fetcher_state]
def format_lsu_state(lsu_state: str) -> str:
lsu_state_map = {
"00": "IDLE",
"01": "REQUESTING",
"10": "WAITING",
"11": "DONE"
}
return lsu_state_map[lsu_state]
def format_memory_controller_state(controller_state: str) -> str:
controller_state_map = {
"000": "IDLE",
"010": "READ_WAITING",
"011": "WRITE_WAITING",
"100": "READ_RELAYING",
"101": "WRITE_RELAYING"
}
return controller_state_map[controller_state]
def format_registers(registers: List[str]) -> str:
formatted_registers = []
for i, reg_value in enumerate(registers):
decimal_value = int(reg_value, 2) # Convert binary string to decimal
reg_idx = 15 - i # Register data is provided in reverse order
formatted_registers.append(f"{format_register(reg_idx)} = {decimal_value}")
formatted_registers.reverse()
return ', '.join(formatted_registers)
def format_cycle(dut, cycle_id: int, thread_id: Optional[int] = None):
logger.debug(f"\n================================== Cycle {cycle_id} ==================================")
for core in dut.cores:
# Not exactly accurate, but good enough for now
if int(str(dut.thread_count.value), 2) <= core.i.value * dut.THREADS_PER_BLOCK.value:
continue
logger.debug(f"\n+--------------------- Core {core.i.value} ---------------------+")
instruction = str(core.core_instance.instruction.value)
for thread in core.core_instance.threads:
if int(thread.i.value) < int(str(core.core_instance.thread_count.value), 2): # if enabled
block_idx = core.core_instance.block_id.value
block_dim = int(core.core_instance.THREADS_PER_BLOCK)
thread_idx = thread.register_instance.THREAD_ID.value
idx = block_idx * block_dim + thread_idx
rs = int(str(thread.register_instance.rs.value), 2)
rt = int(str(thread.register_instance.rt.value), 2)
reg_input_mux = int(str(core.core_instance.decoded_reg_input_mux.value), 2)
alu_out = int(str(thread.alu_instance.alu_out.value), 2)
lsu_out = int(str(thread.lsu_instance.lsu_out.value), 2)
constant = int(str(core.core_instance.decoded_immediate.value), 2)
if (thread_id is None or thread_id == idx):
logger.debug(f"\n+-------- Thread {idx} --------+")
logger.debug("PC:", int(str(core.core_instance.current_pc.value), 2))
logger.debug("Instruction:", format_instruction(instruction))
logger.debug("Core State:", format_core_state(str(core.core_instance.core_state.value)))
logger.debug("Fetcher State:", format_fetcher_state(str(core.core_instance.fetcher_state.value)))
logger.debug("LSU State:", format_lsu_state(str(thread.lsu_instance.lsu_state.value)))
logger.debug("Registers:", format_registers([str(item.value) for item in thread.register_instance.registers]))
logger.debug(f"RS = {rs}, RT = {rt}")
if reg_input_mux == 0:
logger.debug("ALU Out:", alu_out)
if reg_input_mux == 1:
logger.debug("LSU Out:", lsu_out)
if reg_input_mux == 2:
logger.debug("Constant:", constant)
logger.debug("Core Done:", str(core.core_instance.done.value))
================================================
FILE: test/helpers/logger.py
================================================
import datetime
class Logger:
def __init__(self, level="debug"):
self.filename = f"test/logs/log_{datetime.datetime.now().strftime('%Y%m%d%H%M%S')}.txt"
self.level = level
def debug(self, *messages):
if self.level == "debug":
self.info(*messages)
def info(self, *messages):
full_message = ' '.join(str(message) for message in messages)
with open(self.filename, "a") as log_file:
log_file.write(full_message + "\n")
logger = Logger(level="debug")
================================================
FILE: test/helpers/memory.py
================================================
from typing import List
from .logger import logger
class Memory:
def __init__(self, dut, addr_bits, data_bits, channels, name):
self.dut = dut
self.addr_bits = addr_bits
self.data_bits = data_bits
self.memory = [0] * (2**addr_bits)
self.channels = channels
self.name = name
self.mem_read_valid = getattr(dut, f"{name}_mem_read_valid")
self.mem_read_address = getattr(dut, f"{name}_mem_read_address")
self.mem_read_ready = getattr(dut, f"{name}_mem_read_ready")
self.mem_read_data = getattr(dut, f"{name}_mem_read_data")
if name != "program":
self.mem_write_valid = getattr(dut, f"{name}_mem_write_valid")
self.mem_write_address = getattr(dut, f"{name}_mem_write_address")
self.mem_write_data = getattr(dut, f"{name}_mem_write_data")
self.mem_write_ready = getattr(dut, f"{name}_mem_write_ready")
def run(self):
mem_read_valid = [
int(str(self.mem_read_valid.value)[i:i+1], 2)
for i in range(0, len(str(self.mem_read_valid.value)), 1)
]
mem_read_address = [
int(str(self.mem_read_address.value)[i:i+self.addr_bits], 2)
for i in range(0, len(str(self.mem_read_address.value)), self.addr_bits)
]
mem_read_ready = [0] * self.channels
mem_read_data = [0] * self.channels
for i in range(self.channels):
if mem_read_valid[i] == 1:
mem_read_data[i] = self.memory[mem_read_address[i]]
mem_read_ready[i] = 1
else:
mem_read_ready[i] = 0
self.mem_read_data.value = int(''.join(format(d, '0' + str(self.data_bits) + 'b') for d in mem_read_data), 2)
self.mem_read_ready.value = int(''.join(format(r, '01b') for r in mem_read_ready), 2)
if self.name != "program":
mem_write_valid = [
int(str(self.mem_write_valid.value)[i:i+1], 2)
for i in range(0, len(str(self.mem_write_valid.value)), 1)
]
mem_write_address = [
int(str(self.mem_write_address.value)[i:i+self.addr_bits], 2)
for i in range(0, len(str(self.mem_write_address.value)), self.addr_bits)
]
mem_write_data = [
int(str(self.mem_write_data.value)[i:i+self.data_bits], 2)
for i in range(0, len(str(self.mem_write_data.value)), self.data_bits)
]
mem_write_ready = [0] * self.channels
for i in range(self.channels):
if mem_write_valid[i] == 1:
self.memory[mem_write_address[i]] = mem_write_data[i]
mem_write_ready[i] = 1
else:
mem_write_ready[i] = 0
self.mem_write_ready.value = int(''.join(format(w, '01b') for w in mem_write_ready), 2)
def write(self, address, data):
if address < len(self.memory):
self.memory[address] = data
def load(self, rows: List[int]):
for address, data in enumerate(rows):
self.write(address, data)
def display(self, rows, decimal=True):
logger.info("\n")
logger.info(f"{self.name.upper()} MEMORY")
table_size = (8 * 2) + 3
logger.info("+" + "-" * (table_size - 3) + "+")
header = "| Addr | Data "
logger.info(header + " " * (table_size - len(header) - 1) + "|")
logger.info("+" + "-" * (table_size - 3) + "+")
for i, data in enumerate(self.memory):
if i < rows:
if decimal:
row = f"| {i:<4} | {data:<4}"
logger.info(row + " " * (table_size - len(row) - 1) + "|")
else:
data_bin = format(data, f'0{16}b')
row = f"| {i:<4} | {data_bin} |"
logger.info(row + " " * (table_size - len(row) - 1) + "|")
logger.info("+" + "-" * (table_size - 3) + "+")
================================================
FILE: test/helpers/setup.py
================================================
from typing import List
import cocotb
from cocotb.clock import Clock
from cocotb.triggers import RisingEdge
from .memory import Memory
async def setup(
dut,
program_memory: Memory,
program: List[int],
data_memory: Memory,
data: List[int],
threads: int
):
# Setup Clock
clock = Clock(dut.clk, 25, units="us")
cocotb.start_soon(clock.start())
# Reset
dut.reset.value = 1
await RisingEdge(dut.clk)
dut.reset.value = 0
# Load Program Memory
program_memory.load(program)
# Load Data Memory
data_memory.load(data)
# Device Control Register
dut.device_control_write_enable.value = 1
dut.device_control_data.value = threads
await RisingEdge(dut.clk)
dut.device_control_write_enable.value = 0
# Start
dut.start.value = 1
================================================
FILE: test/logs/.gitkeep
================================================
================================================
FILE: test/test_matadd.py
================================================
import cocotb
from cocotb.triggers import RisingEdge
from .helpers.setup import setup
from .helpers.memory import Memory
from .helpers.format import format_cycle
from .helpers.logger import logger
@cocotb.test()
async def test_matadd(dut):
# Program Memory
program_memory = Memory(dut=dut, addr_bits=8, data_bits=16, channels=1, name="program")
program = [
0b0101000011011110, # MUL R0, %blockIdx, %blockDim
0b0011000000001111, # ADD R0, R0, %threadIdx ; i = blockIdx * blockDim + threadIdx
0b1001000100000000, # CONST R1, #0 ; baseA (matrix A base address)
0b1001001000001000, # CONST R2, #8 ; baseB (matrix B base address)
0b1001001100010000, # CONST R3, #16 ; baseC (matrix C base address)
0b0011010000010000, # ADD R4, R1, R0 ; addr(A[i]) = baseA + i
0b0111010001000000, # LDR R4, R4 ; load A[i] from global memory
0b0011010100100000, # ADD R5, R2, R0 ; addr(B[i]) = baseB + i
0b0111010101010000, # LDR R5, R5 ; load B[i] from global memory
0b0011011001000101, # ADD R6, R4, R5 ; C[i] = A[i] + B[i]
0b0011011100110000, # ADD R7, R3, R0 ; addr(C[i]) = baseC + i
0b1000000001110110, # STR R7, R6 ; store C[i] in global memory
0b1111000000000000, # RET ; end of kernel
]
# Data Memory
data_memory = Memory(dut=dut, addr_bits=8, data_bits=8, channels=4, name="data")
data = [
0, 1, 2, 3, 4, 5, 6, 7, # Matrix A (1 x 8)
0, 1, 2, 3, 4, 5, 6, 7 # Matrix B (1 x 8)
]
# Device Control
threads = 8
await setup(
dut=dut,
program_memory=program_memory,
program=program,
data_memory=data_memory,
data=data,
threads=threads
)
data_memory.display(24)
cycles = 0
while dut.done.value != 1:
data_memory.run()
program_memory.run()
await cocotb.triggers.ReadOnly()
format_cycle(dut, cycles)
await RisingEdge(dut.clk)
cycles += 1
logger.info(f"Completed in {cycles} cycles")
data_memory.display(24)
expected_results = [a + b for a, b in zip(data[0:8], data[8:16])]
for i, expected in enumerate(expected_results):
result = data_memory.memory[i + 16]
assert result == expected, f"Result mismatch at index {i}: expected {expected}, got {result}"
================================================
FILE: test/test_matmul.py
================================================
import cocotb
from cocotb.triggers import RisingEdge
from .helpers.setup import setup
from .helpers.memory import Memory
from .helpers.format import format_cycle
from .helpers.logger import logger
@cocotb.test()
async def test_matadd(dut):
# Program Memory
program_memory = Memory(dut=dut, addr_bits=8, data_bits=16, channels=1, name="program")
program = [
0b0101000011011110, # MUL R0, %blockIdx, %blockDim
0b0011000000001111, # ADD R0, R0, %threadIdx ; i = blockIdx * blockDim + threadIdx
0b1001000100000001, # CONST R1, #1 ; increment
0b1001001000000010, # CONST R2, #2 ; N (matrix inner dimension)
0b1001001100000000, # CONST R3, #0 ; baseA (matrix A base address)
0b1001010000000100, # CONST R4, #4 ; baseB (matrix B base address)
0b1001010100001000, # CONST R5, #8 ; baseC (matrix C base address)
0b0110011000000010, # DIV R6, R0, R2 ; row = i // N
0b0101011101100010, # MUL R7, R6, R2
0b0100011100000111, # SUB R7, R0, R7 ; col = i % N
0b1001100000000000, # CONST R8, #0 ; acc = 0
0b1001100100000000, # CONST R9, #0 ; k = 0
# LOOP:
0b0101101001100010, # MUL R10, R6, R2
0b0011101010101001, # ADD R10, R10, R9
0b0011101010100011, # ADD R10, R10, R3 ; addr(A[i]) = row * N + k + baseA
0b0111101010100000, # LDR R10, R10 ; load A[i] from global memory
0b0101101110010010, # MUL R11, R9, R2
0b0011101110110111, # ADD R11, R11, R7
0b0011101110110100, # ADD R11, R11, R4 ; addr(B[i]) = k * N + col + baseB
0b0111101110110000, # LDR R11, R11 ; load B[i] from global memory
0b0101110010101011, # MUL R12, R10, R11
0b0011100010001100, # ADD R8, R8, R12 ; acc = acc + A[i] * B[i]
0b0011100110010001, # ADD R9, R9, R1 ; increment k
0b0010000010010010, # CMP R9, R2
0b0001100000001100, # BRn LOOP ; loop while k < N
0b0011100101010000, # ADD R9, R5, R0 ; addr(C[i]) = baseC + i
0b1000000010011000, # STR R9, R8 ; store C[i] in global memory
0b1111000000000000 # RET ; end of kernel
]
# Data Memory
data_memory = Memory(dut=dut, addr_bits=8, data_bits=8, channels=4, name="data")
data = [
1, 2, 3, 4, # Matrix A (2 x 2)
1, 2, 3, 4, # Matrix B (2 x 2)
]
# Device Control
threads = 4
await setup(
dut=dut,
program_memory=program_memory,
program=program,
data_memory=data_memory,
data=data,
threads=threads
)
data_memory.display(12)
cycles = 0
while dut.done.value != 1:
data_memory.run()
program_memory.run()
await cocotb.triggers.ReadOnly()
format_cycle(dut, cycles, thread_id=1)
await RisingEdge(dut.clk)
cycles += 1
logger.info(f"Completed in {cycles} cycles")
data_memory.display(12)
# Assuming the matrices are 2x2 and the result is stored starting at address 9
matrix_a = [data[0:2], data[2:4]] # First matrix (2x2)
matrix_b = [data[4:6], data[6:8]] # Second matrix (2x2)
expected_results = [
matrix_a[0][0] * matrix_b[0][0] + matrix_a[0][1] * matrix_b[1][0], # C[0,0]
matrix_a[0][0] * matrix_b[0][1] + matrix_a[0][1] * matrix_b[1][1], # C[0,1]
matrix_a[1][0] * matrix_b[0][0] + matrix_a[1][1] * matrix_b[1][0], # C[1,0]
matrix_a[1][0] * matrix_b[0][1] + matrix_a[1][1] * matrix_b[1][1], # C[1,1]
]
for i, expected in enumerate(expected_results):
result = data_memory.memory[i + 8] # Results start at address 9
assert result == expected, f"Result mismatch at index {i}: expected {expected}, got {result}"
gitextract_a16b4xwm/
├── .gitignore
├── Makefile
├── README.md
├── gds/
│ ├── 0/
│ │ └── gpu.gds
│ └── 1/
│ └── gpu.gds
├── src/
│ ├── alu.sv
│ ├── controller.sv
│ ├── core.sv
│ ├── dcr.sv
│ ├── decoder.sv
│ ├── dispatch.sv
│ ├── fetcher.sv
│ ├── gpu.sv
│ ├── lsu.sv
│ ├── pc.sv
│ ├── registers.sv
│ └── scheduler.sv
└── test/
├── __init__.py
├── helpers/
│ ├── format.py
│ ├── logger.py
│ ├── memory.py
│ └── setup.py
├── logs/
│ └── .gitkeep
├── test_matadd.py
└── test_matmul.py
SYMBOL INDEX (21 symbols across 6 files)
FILE: test/helpers/format.py
function format_register (line 4) | def format_register(register: int) -> str:
function format_instruction (line 14) | def format_instruction(instruction: str) -> str:
function format_core_state (line 48) | def format_core_state(core_state: str) -> str:
function format_fetcher_state (line 61) | def format_fetcher_state(fetcher_state: str) -> str:
function format_lsu_state (line 69) | def format_lsu_state(lsu_state: str) -> str:
function format_memory_controller_state (line 78) | def format_memory_controller_state(controller_state: str) -> str:
function format_registers (line 88) | def format_registers(registers: List[str]) -> str:
function format_cycle (line 97) | def format_cycle(dut, cycle_id: int, thread_id: Optional[int] = None):
FILE: test/helpers/logger.py
class Logger (line 3) | class Logger:
method __init__ (line 4) | def __init__(self, level="debug"):
method debug (line 8) | def debug(self, *messages):
method info (line 12) | def info(self, *messages):
FILE: test/helpers/memory.py
class Memory (line 4) | class Memory:
method __init__ (line 5) | def __init__(self, dut, addr_bits, data_bits, channels, name):
method run (line 24) | def run(self):
method write (line 71) | def write(self, address, data):
method load (line 75) | def load(self, rows: List[int]):
method display (line 79) | def display(self, rows, decimal=True):
FILE: test/helpers/setup.py
function setup (line 7) | async def setup(
FILE: test/test_matadd.py
function test_matadd (line 9) | async def test_matadd(dut):
FILE: test/test_matmul.py
function test_matadd (line 9) | async def test_matadd(dut):
Condensed preview — 25 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (94K chars).
[
{
"path": ".gitignore",
"chars": 82,
"preview": "*/**/__pycache__\nbuild/\ntest/logs/*\n!.gitkeep\ngds/**/*.gltf\n\n.DS_Store\nresults.xml"
},
{
"path": "Makefile",
"chars": 587,
"preview": ".PHONY: test compile\n\nexport LIBPYTHON_LOC=$(shell cocotb-config --libpython)\n\ntest_%:\n\tmake compile\n\tiverilog -o build/"
},
{
"path": "README.md",
"chars": 20814,
"preview": "# tiny-gpu\n\nA minimal GPU implementation in Verilog optimized for learning about how GPUs work from the ground up.\n\nBuil"
},
{
"path": "src/alu.sv",
"chars": 1989,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// ARITHMETIC-LOGIC UNIT\n// > Executes computations on register values\n// > In"
},
{
"path": "src/controller.sv",
"chars": 6457,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// MEMORY CONTROLLER\n// > Receives memory requests from all cores\n// > Throttl"
},
{
"path": "src/core.sv",
"chars": 7966,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// COMPUTE CORE\n// > Handles processing 1 block at a time\n// > The core also h"
},
{
"path": "src/dcr.sv",
"chars": 829,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// DEVICE CONTROL REGISTER\n// > Used to configure high-level settings\n// > In "
},
{
"path": "src/decoder.sv",
"chars": 5137,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// INSTRUCTION DECODER\n// > Decodes an instruction into the control signals ne"
},
{
"path": "src/dispatch.sv",
"chars": 3405,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// BLOCK DISPATCH\n// > The GPU has one dispatch unit at the top level\n// > Man"
},
{
"path": "src/fetcher.sv",
"chars": 2084,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// INSTRUCTION FETCHER\n// > Retrieves the instruction at the current PC from g"
},
{
"path": "src/gpu.sv",
"chars": 9462,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// GPU\n// > Built to use an external async memory with multi-channel read/writ"
},
{
"path": "src/lsu.sv",
"chars": 3794,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// LOAD-STORE UNIT\n// > Handles asynchronous memory load and store operations "
},
{
"path": "src/pc.sv",
"chars": 2463,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// PROGRAM COUNTER\n// > Calculates the next PC for each thread to update to (b"
},
{
"path": "src/registers.sv",
"chars": 3523,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// REGISTER FILE\n// > Each thread within each core has it's own register file "
},
{
"path": "src/scheduler.sv",
"chars": 4840,
"preview": "`default_nettype none\n`timescale 1ns/1ns\n\n// SCHEDULER\n// > Manages the entire control flow of a single compute core pro"
},
{
"path": "test/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "test/helpers/format.py",
"chars": 5486,
"preview": "from typing import List, Optional\nfrom .logger import logger\n\ndef format_register(register: int) -> str:\n if register"
},
{
"path": "test/helpers/logger.py",
"chars": 525,
"preview": "import datetime\n\nclass Logger:\n def __init__(self, level=\"debug\"):\n self.filename = f\"test/logs/log_{datetime."
},
{
"path": "test/helpers/memory.py",
"chars": 4044,
"preview": "from typing import List\nfrom .logger import logger\n\nclass Memory:\n def __init__(self, dut, addr_bits, data_bits, chan"
},
{
"path": "test/helpers/setup.py",
"chars": 816,
"preview": "from typing import List\nimport cocotb\nfrom cocotb.clock import Clock\nfrom cocotb.triggers import RisingEdge\nfrom .memory"
},
{
"path": "test/logs/.gitkeep",
"chars": 0,
"preview": ""
},
{
"path": "test/test_matadd.py",
"chars": 2565,
"preview": "import cocotb\nfrom cocotb.triggers import RisingEdge\nfrom .helpers.setup import setup\nfrom .helpers.memory import Memory"
},
{
"path": "test/test_matmul.py",
"chars": 4093,
"preview": "import cocotb\nfrom cocotb.triggers import RisingEdge\nfrom .helpers.setup import setup\nfrom .helpers.memory import Memory"
}
]
// ... and 2 more files (download for full content)
About this extraction
This page contains the full source code of the adam-maj/tiny-gpu GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 25 files (32.4 MB), approximately 22.2k tokens, and a symbol index with 21 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.