Learn
A1.1.3 | CPU AND GPU DIFFERENCES (HL ONLY)
01 | THE QUESTION IS WHICH WORK SUITS WHICH DESIGN
A robot might read controls, decide whether to stop and classify a batch of camera images. These are all processing tasks, but their structure is different. A short sequence of dependent decisions is not the same workload as millions of related numerical operations.
The CPU and GPU both execute instructions and perform calculations. Their different designs favour different kinds of work. A CPU is designed for flexible general-purpose processing with strong performance on individual instruction streams. A GPU devotes substantial resources to high-throughput processing of suitable parallel workloads.
For this HL-only objective, move beyond listing “CPU: general; GPU: graphics”. Explain how architecture affects processing, memory access and energy use, then connect those differences to an application.
There is no universal winner. A useful system often combines both, with software dividing work and coordinating access to data. The best choice depends on the whole task, not the largest advertised core count.
02 | LATENCY AND THROUGHPUT ARE DIFFERENT GOALS
Latency is the time taken to complete a particular operation or request. If a robot must react to an obstacle, the delay before that decision is available matters. Throughput is the amount of work completed per unit of time, such as images classified per second.
A CPU core typically includes sophisticated control and execution machinery to make progress through a varied instruction stream quickly. A GPU is designed to keep many arithmetic resources busy across a large amount of work, even if an individual task does not finish sooner.
Think of answering one difficult question versus marking a large batch of very similar calculations. Improving the time for one answer and improving the number completed each second are related but different objectives.
Both processors can support parallelism. Modern CPUs commonly contain several cores and vector instructions, while GPUs still contain control logic and execute sequences of instructions. Avoid treating “CPU = only serial” or “GPU = no sequential steps” as literal rules.
03 | CORE ARCHITECTURE: FLEXIBILITY AND MANY ARITHMETIC RESOURCES
A CPU generally has a smaller number of comparatively complex general-purpose cores. Features such as branch prediction, out-of-order execution and caches help sustain progress through varied instructions and memory accesses. Exact features depend on the processor.
A branch is a decision that can select a different instruction path. Branch prediction estimates the likely path so useful work can be prepared early. Out-of-order execution allows eligible operations to proceed before earlier operations have completed, while preserving the required program behaviour.
A GPU typically has many arithmetic execution resources organised into groups, with scheduling and shared control structures supporting large numbers of lightweight threads. A thread is a sequence of execution, not a physical core.
The word core is used differently in CPU and GPU specifications. One advertised GPU arithmetic core is not a complete CPU core with identical control, caching and execution resources. Comparing counts directly does not establish performance.
CPU | FLEXIBLE INSTRUCTION PROCESSING
Rich control, prediction, execution and cache resources support varied work. The pictured two cores are illustrative.
GPU | HIGH PARALLEL THROUGHPUT
Grouped arithmetic resources, scheduling and memory support many similar tasks. These six boxes are illustrative, not a core count.
04 | CONTROL FLOW AND DATA PARALLELISM
An operating system, user interface or complex decision process can involve varied instructions, frequent choices and dependencies. CPU designs are well suited to handling this mixture efficiently. Several CPU cores can run separate suitable tasks or threads.
A large image filter instead applies a similar rule across many pixels. A GPU can schedule groups of threads to carry out related operations on different values. This is data parallelism. Matrix operations in machine learning offer another important example.
In common GPU execution models, threads in a group share instruction issue. If they take different branches, the hardware may need to handle different paths separately, reducing the fraction of resources doing useful work at a given moment. This is often called branch divergence.
Divergence does not mean the GPU cannot use IF statements. It means the structure of the decisions can affect efficiency. Likewise, a dependency does not prove that no parallel algorithm exists; it may require redesign rather than a simple division into independent items.
05 | PROCESSING POWER IS WORKLOAD-DEPENDENT
Processing power is meaningful only in relation to work and conditions. A GPU may sustain a high rate of suitable arithmetic operations, but that does not guarantee low completion time for a short dependent program.
A CPU might respond well to a small task requiring several unpredictable decisions. A GPU might complete a large, regular matrix calculation much sooner because it can exploit far more useful parallel work.
Performance also depends on numerical precision, memory access, software implementation and available data. Peak arithmetic rates describe particular operation types under favourable assumptions; they are not a measurement of every application.
Compare the same required result and accuracy using an appropriate measure, such as end-to-end completion time or completed items per second. A faster arithmetic stage can still leave the overall application limited by data transfers or a sequential stage.
06 | MEMORY ACCESS: CACHES, BANDWIDTH AND WAITING
Both CPUs and GPUs have registers, caches and access to larger memory. The difference is not “CPU has cache; GPU has none”. CPU cache hierarchies help reduce the average delay of accessing instructions and data, especially when recently used or nearby information is reused.
GPUs combine substantial memory bandwidth with many ready execution tasks. When one group waits for data, another eligible group can run, helping hide some of the waiting time. This requires enough ready work and sufficient execution resources; the latency has not vanished.
On suitable GPU hardware, related accesses by neighbouring threads can be combined into fewer memory transactions. Organising data for these access patterns can improve efficiency. Scattered accesses, insufficient reuse or excessive transfers can reduce the benefit.
Memory bandwidth is a transfer rate, while memory latency is the delay before an access is served. Capacity is how much data can be held. These are three different properties, and none is interchangeable with the number of processing cores.
07 | SEPARATE AND SHARED MEMORY ARRANGEMENTS
A discrete GPU commonly has its own device memory, while the CPU accesses system memory. Software may need to transfer input data to the GPU and return results. The connection between them can become a bottleneck even if the GPU’s own memory interface is fast.
An integrated GPU commonly shares physical memory with CPU resources. Some platforms also provide a unified memory programming model, managing access or movement on the programmer’s behalf. Shared or unified arrangements do not make every access free or remove all coordination.
If several GPU stages use the same data, keeping it available to the GPU between stages can avoid repeated transfers. Returning a tiny intermediate result to the CPU after every operation may be slower than retaining a larger chain of suitable processing on the GPU.
The correct description therefore depends on the system. Do not claim that every CPU–GPU pair has separate physical memories, or that a shared-memory system has no bandwidth limits or synchronisation requirements.
| Issue | Typical CPU emphasis | Typical GPU emphasis |
|---|---|---|
| Useful memory behaviour | Use caches and execution machinery to reduce or tolerate delays in varied work. | Supply high-bandwidth data and schedule many ready threads to sustain throughput. |
| Access pattern | Locality still matters, even for general-purpose code. | Grouped, organised accesses can make good use of the memory interface. |
| Where data resides | Usually uses system memory through its cache hierarchy. | Discrete devices often use device memory; integrated designs commonly share memory. |
| Main caution | Caches do not guarantee every access is fast. | Large bandwidth does not eliminate transfer overhead or dependencies. |
08 | THE WHOLE-JOB COST OF OFFLOADING
Offloading means assigning a portion of the application’s work to another processing resource. The CPU and software prepare a suitable GPU task, arrange access to input data, launch the work and eventually use its output.
Consider an invented example: the CPU takes 100 ms to complete a calculation. The GPU calculation itself takes 10 ms, but preparing and transferring data adds 120 ms. The GPU route takes 130 ms overall, so the faster calculation stage has not made the job faster.
If a much larger workload takes 1,000 ms on the CPU and only 40 ms for the GPU calculation, the same 120 ms overhead produces 160 ms overall. Now the end-to-end result is faster. These values are examples, not hardware measurements.
A repeated workflow can change the balance. If input data already resides on the GPU or several stages reuse it there, some transfer overhead may be avoided. Explain what is included in the measured time before claiming a speed-up.
09 | TRY A PERFORMANCE AND ENERGY MODEL
This model separates the CPU-only time into a sequential part and a part that can be accelerated. Only the eligible parallel part receives the chosen acceleration factor. The CPU still handles the sequential part.
Change the values to explore when the hybrid route helps. The timings, acceleration factors and power values are invented teaching assumptions. The model is not a benchmark or a hardware recommendation.
It assumes sequential execution of the displayed stages, no overlap and fixed average powers. CPU-only power is 50 W. In the hybrid route, the sequential stage uses 50 W; the offload stage uses 20 W for the host CPU plus 180 W for the GPU. Other system energy is excluded.
CPU-only job
CPU and GPU job
The bars share a time scale. Calculated values below them provide the exact model results.
MODEL FORMULAS
Sequential time = CPU-only time × sequential fraction Eligible CPU time = CPU-only time − sequential time Offload time = overhead + eligible CPU time ÷ acceleration Hybrid time = sequential time + offload time CPU-only energy = 50 × CPU-only time ÷ 1000 joules Hybrid energy = (50 × sequential time + 200 × offload time) ÷ 1000 joules
Try a CPU-only time of 8 ms with zero sequential work, factor 32 and overhead 20 ms. Then try 1,000 ms with the same settings. Finally choose 90% sequential work. Explain why accelerating one part cannot erase the time spent in the other part.
10 | POWER EFFICIENCY: WATTS ARE NOT JOULES
Power is the rate of energy use, measured in watts. Energy is the total used over an interval, measured in joules. For a constant or appropriate average power, energy = power × time. Convert milliseconds to seconds before multiplying by watts.
A GPU can draw more power while operating yet use less total energy for a suitable job if it finishes sufficiently sooner. Conversely, leaving many resources underused while paying transfer costs may give poor energy efficiency. Neither processor type is always the more efficient choice.
For example, a 50 W processor running for 2 seconds uses 100 J. A device using 150 W for 0.4 seconds uses 60 J. This comparison concerns only the stated devices and durations. A complete system comparison must include the host, memory, transfers and other relevant energy.
Use measures such as jobs per joule, joules per completed task or performance per watt with their assumptions stated. Compare equivalent outputs and accuracy. A low power rating alone does not establish the lowest energy per task.
11 | A CPU–GPU WORKFLOW NEEDS COORDINATION
A hybrid application uses different processing resources for different parts of its workload. The CPU may prepare a dataset, submit a GPU operation, perform independent work and later combine the results. The exact division depends on the application.
A GPU kernel is a function executed as many GPU thread instances. Launching a kernel or arranging a data transfer can be asynchronous: the CPU may continue before that GPU work has finished. Asynchronous does not mean the required result is already available.
Synchronisation establishes the necessary ordering or completion before a dependent operation proceeds. If the CPU reads an output before the GPU has finished writing it, the program may use incomplete or incorrect data. Software must arrange the correct dependencies and data access.
Some implementations can overlap independent CPU work, GPU processing and transfers. Such overlap requires suitable work and support; it is not guaranteed just because both processors are present. The interactive model intentionally leaves overlap out so its arithmetic stays transparent.
- CPU: prepare the task and its input data.
- Data access: transfer data or arrange access in the appropriate memory.
- GPU: execute suitable parallel work.
- Completion: ensure the dependent output is ready and accessible.
- CPU or next stage: consume the result and continue.
Independent work may overlap in a more advanced implementation. This sequence shows the dependency that must still be respected.
12 | APPLY THE COMPARISON TO REAL SCENARIOS
| Scenario | CPU contribution | GPU contribution | Why the combination matters |
|---|---|---|---|
| Interactive game | Handle program decisions, input and game coordination. | Perform suitable rendering and numerical calculations. | The next frame depends on both useful graphics work and timely application decisions. |
| Machine learning pipeline | Prepare data, arrange batches and coordinate stages. | Accelerate supported matrix operations for training or inference. | Data preparation or transfers can limit otherwise fast GPU processing. |
| Scientific simulation | Manage model setup, control and less parallel operations. | Process suitable grid or particle calculations. | Dependencies and exchanges between stages must remain correct. |
| Robot vision and control | Handle control decisions and coordinate sensor processing. | Accelerate suitable image or model operations when useful. | High image throughput does not by itself guarantee a timely control response. |
| Small dependent calculation | Execute the short sequence directly. | May offer little useful benefit after overhead. | A large parallel device is not automatically the best route. |
These divisions are examples, not rules that every engine follows. State the workload, identify the design feature that helps, and explain a limit. That produces a stronger answer than assigning a whole industry to one processor.
13 | EXPLAIN WITH A CAUSE AND A CONSEQUENCE
The command explain requires reasons and relationships. Build each point as: design feature → effect on execution → consequence for the workload. Use connectives such as because, therefore and resulting in when they make that link explicit.
For example: “A GPU has many arithmetic resources organised for related parallel work. Because a large matrix calculation contains many similar operations, these resources can be kept busy, resulting in high throughput.”
Then qualify the claim: “However, if the dataset must repeatedly move between separate CPU and GPU memories, transfer time can reduce the overall benefit.”
For CPU work: “Sophisticated control and execution features help a CPU progress through varied instruction paths, so it is well suited to general-purpose tasks with decisions and dependencies.” Avoid claiming that branch prediction is always correct or that CPUs cannot process data in parallel.
14 | COMPARISON SUMMARY AND COMMON TRAPS
| Aspect | CPU design emphasis | GPU design emphasis |
|---|---|---|
| Design goal | Flexible, responsive execution of general-purpose instruction streams. | High throughput across suitable parallel workloads. |
| Core structure | Fewer, more complex general-purpose cores in typical comparisons. | Many grouped arithmetic resources and lightweight execution tasks. |
| Control flow | Sophisticated handling of varied instructions and branches. | Best utilisation often comes from coherent, related work; divergence can reduce efficiency. |
| Memory access | Caches, locality and execution machinery help manage delays. | Bandwidth, organised access and many ready threads help sustain throughput. |
| Processing power | Often strong on varied or dependent work. | Often strong on large amounts of suitable arithmetic. |
| Power efficiency | Can be efficient for small or irregular tasks without offload costs. | Can be efficient for suitable substantial work, despite higher active power in some systems. |
| Cooperation | Coordinate the application and use results as dependencies permit. | Execute delegated work with data access and completion correctly managed. |
Do not equate a GPU thread with a physical core, compare raw CPU/GPU core counts as identical units, or confuse low power with low total energy. Do not assume every CPU–GPU pair has separate memory or that transfers are free when memory is shared.
Use the Questions tab to practise full explanations. The next lesson, A1.1.4, returns to the SL and HL content on primary memory.