IB COMPUTER SCIENCE | HL ONLY | A1.1.3 CPU and GPU Differences (HL only)

Lesson objective

Explain CPU and GPU differences in design, cores, processing, memory access and power efficiency, and explain their coordinated use in a hybrid application.

Learn

A1.1.3 | CPU AND GPU DIFFERENCES (HL ONLY)

01 | THE QUESTION IS WHICH WORK SUITS WHICH DESIGN

A robot might read controls, decide whether to stop and classify a batch of camera images. These are all processing tasks, but their structure is different. A short sequence of dependent decisions is not the same workload as millions of related numerical operations.

The CPU and GPU both execute instructions and perform calculations. Their different designs favour different kinds of work. A CPU is designed for flexible general-purpose processing with strong performance on individual instruction streams. A GPU devotes substantial resources to high-throughput processing of suitable parallel workloads.

For this HL-only objective, move beyond listing “CPU: general; GPU: graphics”. Explain how architecture affects processing, memory access and energy use, then connect those differences to an application.

There is no universal winner. A useful system often combines both, with software dividing work and coordinating access to data. The best choice depends on the whole task, not the largest advertised core count.

02 | LATENCY AND THROUGHPUT ARE DIFFERENT GOALS

Latency is the time taken to complete a particular operation or request. If a robot must react to an obstacle, the delay before that decision is available matters. Throughput is the amount of work completed per unit of time, such as images classified per second.

A CPU core typically includes sophisticated control and execution machinery to make progress through a varied instruction stream quickly. A GPU is designed to keep many arithmetic resources busy across a large amount of work, even if an individual task does not finish sooner.

Think of answering one difficult question versus marking a large batch of very similar calculations. Improving the time for one answer and improving the number completed each second are related but different objectives.

Both processors can support parallelism. Modern CPUs commonly contain several cores and vector instructions, while GPUs still contain control logic and execute sequences of instructions. Avoid treating “CPU = only serial” or “GPU = no sequential steps” as literal rules.

03 | CORE ARCHITECTURE: FLEXIBILITY AND MANY ARITHMETIC RESOURCES

A CPU generally has a smaller number of comparatively complex general-purpose cores. Features such as branch prediction, out-of-order execution and caches help sustain progress through varied instructions and memory accesses. Exact features depend on the processor.

A branch is a decision that can select a different instruction path. Branch prediction estimates the likely path so useful work can be prepared early. Out-of-order execution allows eligible operations to proceed before earlier operations have completed, while preserving the required program behaviour.

A GPU typically has many arithmetic execution resources organised into groups, with scheduling and shared control structures supporting large numbers of lightweight threads. A thread is a sequence of execution, not a physical core.

The word core is used differently in CPU and GPU specifications. One advertised GPU arithmetic core is not a complete CPU core with identical control, caching and execution resources. Comparing counts directly does not establish performance.

TWO DESIGN EMPHASES, NOT REAL CHIP LAYOUTS

CPU | FLEXIBLE INSTRUCTION PROCESSING

Complex general-purpose core
Complex general-purpose core

Rich control, prediction, execution and cache resources support varied work. The pictured two cores are illustrative.

GPU | HIGH PARALLEL THROUGHPUT

ArithmeticArithmeticArithmeticArithmeticArithmeticArithmetic

Grouped arithmetic resources, scheduling and memory support many similar tasks. These six boxes are illustrative, not a core count.

04 | CONTROL FLOW AND DATA PARALLELISM

An operating system, user interface or complex decision process can involve varied instructions, frequent choices and dependencies. CPU designs are well suited to handling this mixture efficiently. Several CPU cores can run separate suitable tasks or threads.

A large image filter instead applies a similar rule across many pixels. A GPU can schedule groups of threads to carry out related operations on different values. This is data parallelism. Matrix operations in machine learning offer another important example.

In common GPU execution models, threads in a group share instruction issue. If they take different branches, the hardware may need to handle different paths separately, reducing the fraction of resources doing useful work at a given moment. This is often called branch divergence.

Divergence does not mean the GPU cannot use IF statements. It means the structure of the decisions can affect efficiency. Likewise, a dependency does not prove that no parallel algorithm exists; it may require redesign rather than a simple division into independent items.

05 | PROCESSING POWER IS WORKLOAD-DEPENDENT

Processing power is meaningful only in relation to work and conditions. A GPU may sustain a high rate of suitable arithmetic operations, but that does not guarantee low completion time for a short dependent program.

A CPU might respond well to a small task requiring several unpredictable decisions. A GPU might complete a large, regular matrix calculation much sooner because it can exploit far more useful parallel work.

Performance also depends on numerical precision, memory access, software implementation and available data. Peak arithmetic rates describe particular operation types under favourable assumptions; they are not a measurement of every application.

Compare the same required result and accuracy using an appropriate measure, such as end-to-end completion time or completed items per second. A faster arithmetic stage can still leave the overall application limited by data transfers or a sequential stage.

06 | MEMORY ACCESS: CACHES, BANDWIDTH AND WAITING

Both CPUs and GPUs have registers, caches and access to larger memory. The difference is not “CPU has cache; GPU has none”. CPU cache hierarchies help reduce the average delay of accessing instructions and data, especially when recently used or nearby information is reused.

GPUs combine substantial memory bandwidth with many ready execution tasks. When one group waits for data, another eligible group can run, helping hide some of the waiting time. This requires enough ready work and sufficient execution resources; the latency has not vanished.

On suitable GPU hardware, related accesses by neighbouring threads can be combined into fewer memory transactions. Organising data for these access patterns can improve efficiency. Scattered accesses, insufficient reuse or excessive transfers can reduce the benefit.

Memory bandwidth is a transfer rate, while memory latency is the delay before an access is served. Capacity is how much data can be held. These are three different properties, and none is interchangeable with the number of processing cores.

07 | SEPARATE AND SHARED MEMORY ARRANGEMENTS

A discrete GPU commonly has its own device memory, while the CPU accesses system memory. Software may need to transfer input data to the GPU and return results. The connection between them can become a bottleneck even if the GPU’s own memory interface is fast.

An integrated GPU commonly shares physical memory with CPU resources. Some platforms also provide a unified memory programming model, managing access or movement on the programmer’s behalf. Shared or unified arrangements do not make every access free or remove all coordination.

If several GPU stages use the same data, keeping it available to the GPU between stages can avoid repeated transfers. Returning a tiny intermediate result to the CPU after every operation may be slower than retaining a larger chain of suitable processing on the GPU.

The correct description therefore depends on the system. Do not claim that every CPU–GPU pair has separate physical memories, or that a shared-memory system has no bandwidth limits or synchronisation requirements.

IssueTypical CPU emphasisTypical GPU emphasis
Useful memory behaviourUse caches and execution machinery to reduce or tolerate delays in varied work.Supply high-bandwidth data and schedule many ready threads to sustain throughput.
Access patternLocality still matters, even for general-purpose code.Grouped, organised accesses can make good use of the memory interface.
Where data residesUsually uses system memory through its cache hierarchy.Discrete devices often use device memory; integrated designs commonly share memory.
Main cautionCaches do not guarantee every access is fast.Large bandwidth does not eliminate transfer overhead or dependencies.

08 | THE WHOLE-JOB COST OF OFFLOADING

Offloading means assigning a portion of the application’s work to another processing resource. The CPU and software prepare a suitable GPU task, arrange access to input data, launch the work and eventually use its output.

Consider an invented example: the CPU takes 100 ms to complete a calculation. The GPU calculation itself takes 10 ms, but preparing and transferring data adds 120 ms. The GPU route takes 130 ms overall, so the faster calculation stage has not made the job faster.

If a much larger workload takes 1,000 ms on the CPU and only 40 ms for the GPU calculation, the same 120 ms overhead produces 160 ms overall. Now the end-to-end result is faster. These values are examples, not hardware measurements.

A repeated workflow can change the balance. If input data already resides on the GPU or several stages reuse it there, some transfer overhead may be avoided. Explain what is included in the measured time before claiming a speed-up.

09 | TRY A PERFORMANCE AND ENERGY MODEL

This model separates the CPU-only time into a sequential part and a part that can be accelerated. Only the eligible parallel part receives the chosen acceleration factor. The CPU still handles the sequential part.

Change the values to explore when the hybrid route helps. The timings, acceleration factors and power values are invented teaching assumptions. The model is not a benchmark or a hardware recommendation.

It assumes sequential execution of the displayed stages, no overlap and fixed average powers. CPU-only power is 50 W. In the hybrid route, the sequential stage uses 50 W; the offload stage uses 20 W for the host CPU plus 180 W for the GPU. Other system energy is excluded.

CPU-only job

CPU and GPU job

The bars share a time scale. Calculated values below them provide the exact model results.

MODEL FORMULAS

Sequential time = CPU-only time × sequential fraction
Eligible CPU time = CPU-only time − sequential time
Offload time = overhead + eligible CPU time ÷ acceleration
Hybrid time = sequential time + offload time
CPU-only energy = 50 × CPU-only time ÷ 1000 joules
Hybrid energy = (50 × sequential time + 200 × offload time) ÷ 1000 joules

Try a CPU-only time of 8 ms with zero sequential work, factor 32 and overhead 20 ms. Then try 1,000 ms with the same settings. Finally choose 90% sequential work. Explain why accelerating one part cannot erase the time spent in the other part.

10 | POWER EFFICIENCY: WATTS ARE NOT JOULES

Power is the rate of energy use, measured in watts. Energy is the total used over an interval, measured in joules. For a constant or appropriate average power, energy = power × time. Convert milliseconds to seconds before multiplying by watts.

A GPU can draw more power while operating yet use less total energy for a suitable job if it finishes sufficiently sooner. Conversely, leaving many resources underused while paying transfer costs may give poor energy efficiency. Neither processor type is always the more efficient choice.

For example, a 50 W processor running for 2 seconds uses 100 J. A device using 150 W for 0.4 seconds uses 60 J. This comparison concerns only the stated devices and durations. A complete system comparison must include the host, memory, transfers and other relevant energy.

Use measures such as jobs per joule, joules per completed task or performance per watt with their assumptions stated. Compare equivalent outputs and accuracy. A low power rating alone does not establish the lowest energy per task.

11 | A CPU–GPU WORKFLOW NEEDS COORDINATION

A hybrid application uses different processing resources for different parts of its workload. The CPU may prepare a dataset, submit a GPU operation, perform independent work and later combine the results. The exact division depends on the application.

A GPU kernel is a function executed as many GPU thread instances. Launching a kernel or arranging a data transfer can be asynchronous: the CPU may continue before that GPU work has finished. Asynchronous does not mean the required result is already available.

Synchronisation establishes the necessary ordering or completion before a dependent operation proceeds. If the CPU reads an output before the GPU has finished writing it, the program may use incomplete or incorrect data. Software must arrange the correct dependencies and data access.

Some implementations can overlap independent CPU work, GPU processing and transfers. Such overlap requires suitable work and support; it is not guaranteed just because both processors are present. The interactive model intentionally leaves overlap out so its arithmetic stays transparent.

A SIMPLE CORRECTLY ORDERED WORKFLOW
  1. CPU: prepare the task and its input data.
  2. Data access: transfer data or arrange access in the appropriate memory.
  3. GPU: execute suitable parallel work.
  4. Completion: ensure the dependent output is ready and accessible.
  5. CPU or next stage: consume the result and continue.

Independent work may overlap in a more advanced implementation. This sequence shows the dependency that must still be respected.

12 | APPLY THE COMPARISON TO REAL SCENARIOS

ScenarioCPU contributionGPU contributionWhy the combination matters
Interactive gameHandle program decisions, input and game coordination.Perform suitable rendering and numerical calculations.The next frame depends on both useful graphics work and timely application decisions.
Machine learning pipelinePrepare data, arrange batches and coordinate stages.Accelerate supported matrix operations for training or inference.Data preparation or transfers can limit otherwise fast GPU processing.
Scientific simulationManage model setup, control and less parallel operations.Process suitable grid or particle calculations.Dependencies and exchanges between stages must remain correct.
Robot vision and controlHandle control decisions and coordinate sensor processing.Accelerate suitable image or model operations when useful.High image throughput does not by itself guarantee a timely control response.
Small dependent calculationExecute the short sequence directly.May offer little useful benefit after overhead.A large parallel device is not automatically the best route.

These divisions are examples, not rules that every engine follows. State the workload, identify the design feature that helps, and explain a limit. That produces a stronger answer than assigning a whole industry to one processor.

13 | EXPLAIN WITH A CAUSE AND A CONSEQUENCE

The command explain requires reasons and relationships. Build each point as: design feature → effect on execution → consequence for the workload. Use connectives such as because, therefore and resulting in when they make that link explicit.

For example: “A GPU has many arithmetic resources organised for related parallel work. Because a large matrix calculation contains many similar operations, these resources can be kept busy, resulting in high throughput.”

Then qualify the claim: “However, if the dataset must repeatedly move between separate CPU and GPU memories, transfer time can reduce the overall benefit.”

For CPU work: “Sophisticated control and execution features help a CPU progress through varied instruction paths, so it is well suited to general-purpose tasks with decisions and dependencies.” Avoid claiming that branch prediction is always correct or that CPUs cannot process data in parallel.

14 | COMPARISON SUMMARY AND COMMON TRAPS

AspectCPU design emphasisGPU design emphasis
Design goalFlexible, responsive execution of general-purpose instruction streams.High throughput across suitable parallel workloads.
Core structureFewer, more complex general-purpose cores in typical comparisons.Many grouped arithmetic resources and lightweight execution tasks.
Control flowSophisticated handling of varied instructions and branches.Best utilisation often comes from coherent, related work; divergence can reduce efficiency.
Memory accessCaches, locality and execution machinery help manage delays.Bandwidth, organised access and many ready threads help sustain throughput.
Processing powerOften strong on varied or dependent work.Often strong on large amounts of suitable arithmetic.
Power efficiencyCan be efficient for small or irregular tasks without offload costs.Can be efficient for suitable substantial work, despite higher active power in some systems.
CooperationCoordinate the application and use results as dependencies permit.Execute delegated work with data access and completion correctly managed.

Do not equate a GPU thread with a physical core, compare raw CPU/GPU core counts as identical units, or confuse low power with low total energy. Do not assume every CPU–GPU pair has separate memory or that transfers are free when memory is shared.

Use the Questions tab to practise full explanations. The next lesson, A1.1.4, returns to the SL and HL content on primary memory.

Further reading: NVIDIA’s host–device and memory guidance

MATCH THE COMPARISON TERM TO ITS MEANING

Terminology

Terminology

Latency

The time taken to complete an operation or request.

Throughput

The amount of work completed per unit of time.

General-purpose core

A processing core designed to handle a broad range of instruction sequences.

Arithmetic resource

Hardware that carries out numerical or logical operations.

Thread

A sequence of execution, distinct from a physical core.

Branch

A decision selecting a path through instructions.

Branch prediction

Estimating the likely instruction path to help prepare useful work.

Out-of-order execution

Executing eligible operations out of program order while preserving required behaviour.

Data parallelism

Applying similar processing concurrently to different data items.

Branch divergence

Different paths taken by threads in an execution group, potentially reducing efficiency.

Cache

Fast storage retaining copies of useful instructions or data.

Memory latency

The delay involved in serving a memory access.

Memory bandwidth

The rate at which data can be transferred through a memory interface.

Memory capacity

The amount of data a memory can hold.

Locality

The tendency to reuse nearby or recently accessed data or instructions.

Offloading

Assigning part of a workload to another processing resource.

Overhead

Extra time or work needed to arrange, transfer or coordinate processing.

Kernel

A function executed by GPU thread instances.

Host

The CPU-side system coordinating device work in this processing model.

Device memory

Memory associated with the processing device, such as a discrete GPU.

Asynchronous operation

An operation that may continue after the initiating call returns.

Synchronisation

Establishing required ordering or completion between operations.

Power

The rate of energy use, measured in watts.

Energy

The amount used over an interval, measured in joules.

Energy efficiency

Useful work completed relative to the energy required.

Bottleneck

A limiting stage or resource that constrains overall performance.

Questions

Questions

CHECK YOUR UNDERSTANDING

Select all correct choices. Each exact set earns one point.

1. Which statements compare design goals correctly?
2. Which best explains why raw core counts cannot establish CPU–GPU performance?
3. What can reduce efficiency when threads in a GPU execution group take different paths?
4. Which memory statements are accurate?
5. CPU-only time is 100 ms. GPU computation is 10 ms and offload overhead is 120 ms. What follows?
6. Which statements about watts and joules are correct?
7. What should happen before CPU code uses a required GPU result?
8. Which changes can improve a suitable GPU workflow?
9. Which is an explanation rather than a bare claim?
10. If 90% of a model job remains sequential, what does accelerating only the other 10% imply?

EXPLAIN AND COMPARE

Write your answers before revealing each sample.

1. Explain how CPU and GPU design goals differ.

2. Explain why GPU arithmetic cores cannot simply be counted as CPU cores.

3. Explain why a large image filter can benefit from a GPU.

4. Explain one effect of branch divergence.

5. Explain differences in CPU and GPU memory-access strategies.

6. Explain why discrete GPU transfer costs can change the best processor choice.

7. Explain how higher power can still mean lower energy for a task.

8. Explain why synchronisation is needed in a CPU–GPU application.

9. Explain how keeping intermediate data on a GPU can improve performance.

10. Explain why a short dependency-heavy robot control decision may remain on the CPU.

APPLY IT | A MIXED ROBOTICS WORKLOAD

A robot needs a short obstacle-stop decision and a large batch of image classifications. Explain a sensible CPU–GPU task division. Address architecture, data movement, response time, completion checks and energy per job. State which measurements would be needed before claiming a real performance improvement.

Flashcards

Flashcards

Click to flip. Select the ideas you need to revisit.

0 cards selected for revision.

    Selections are kept while this page is open.

    Workbook

    Workbook

    COMING SOON

    The workbook for A1.1.3 CPU and GPU Differences (HL only) is coming soon. Complete the matching task and written questions in your exercise book.