Skip to content

fix(vt-d): invalidate cache when initializing and updating#665

Open
ytakano wants to merge 17 commits intotier4:mainfrom
ytakano:fix_interrupt_remapping3
Open

fix(vt-d): invalidate cache when initializing and updating#665
ytakano wants to merge 17 commits intotier4:mainfrom
ytakano:fix_interrupt_remapping3

Conversation

@ytakano
Copy link
Collaborator

@ytakano ytakano commented Jan 13, 2026 •
edited
Loading

Description

VT-d Interrupt Remapping Implementation and Enhancements.
Invalidate the cache of interrupt remapping when initializing and updating.

Cache Invalidation

  • Implemented global interrupt entry cache invalidation with ESIRTPS (Enhanced Set Interrupt Root Table Pointer Support) detection
  • Automatic invalidation for hardware without ESIRTPS support
スクリーンショット 2026-01-06 17 41 20
  • Invalidate cache when updating interrupt remappings
スクリーンショット 2026-01-06 17 36 08

IOMMU State Management

  • Added IommuInfo and VtdUnit structures.
  • Added IOMMU global variable to track both VT-d units and interrupt remapping tables

Related links

https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://cdrdv2-public.intel.com/671081/vt-directed-io-spec.pdf&ved=2ahUKEwjJ1oCiufaRAxWPnq8BHVbbLy8QFnoECAwQAQ&usg=AOvVaw3fO9wnba58SqsP2HNKwdfu

How was this PR tested?

Tested on a physical x86_64 machine.

[          538 INFO] Vt-d Queued Invalidation enabled: Segment = 0, Queue PhyAddr = 0x44a6000, IQA = 0x44a6000, IQH = 0
, IQT = 0
[          571 INFO] Vt-d Interrupt Remapping: Segment = 0, Vt-d Base = 0xfed90000, Table PhyAddr = 0x43a5000, enabled
= true, mode = x2APIC, extended capability = 0x29a00f0505e, global status = 0x47000000, IRTA = 0x43a580f, DHRD.flags =
0x0
Vt-d Device Scope: entry type = 1, length = 8, flags = 0x0, enumeration ID = 0, start bus number = 0, path = [2, 0]
[          665 INFO] Vt-d Queued Invalidation enabled: Segment = 0, Queue PhyAddr = 0x44a8000, IQA = 0x44a8000, IQH = 0
, IQT = 0
[          698 INFO] Vt-d Interrupt Remapping: Segment = 0, Vt-d Base = 0xfed91000, Table PhyAddr = 0x43a5000, enabled
= true, mode = x2APIC, extended capability = 0xf050da, global status = 0x47000000, IRTA = 0x43a580f, DHRD.flags = 0x1
Vt-d Device Scope: entry type = 3, length = 8, flags = 0x0, enumeration ID = 2, start bus number = 0, path = [1e, 7]
Vt-d Device Scope: entry type = 4, length = 8, flags = 0x0, enumeration ID = 0, start bus number = 0, path = [1e, 6]
[          825 INFO] VT-d segment 0: ESIRTPS not supported (ECAP=0x29a00f0505e), performing global invalidation
[          854 INFO] Vt-d: Invalidated all interrupt entries: segment = 0, vtd_base = 0x100fed90000, iq_base = 0x400001
02000, IQH = 0x20, IQT = 0x20
[          893 INFO] VT-d segment 0: ESIRTPS not supported (ECAP=0xf050da), performing global invalidation
[          921 INFO] Vt-d: Invalidated all interrupt entries: segment = 0, vtd_base = 0x100fed91000, iq_base = 0x400001
04000, IQH = 0x20, IQT = 0x20
[          960 INFO] Vt-d IOMMU Info:
VT-d Segment 0:
  VT-d Unit [0]: VT-d Base = 0x100fed90000, IQ Base = 0x40000102000, Index = 0
  VT-d Unit [1]: VT-d Base = 0x100fed91000, IQ Base = 0x40000104000, Index = 1
  Interrupt Remapping Table: Base = 0x40000001000, is_x2apic = true

Notes for reviewers

ytakano added 17 commits January 6, 2026 13:38
@ytakano
fix(interrupt remap): fix the initialization of interrupt remapping
8b78f39 
1. do not use the global status register value for the global command
   register
2. wait untile enabled after updatating the global command register
3. print more registers for debugging

Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
feat(iommu): initialize invalidate queue
b390701 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(iommu): print IQH and IQT of QIE
6e350be 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(iommu): add update_global_command()
5eca6d2 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(iommu): Wait for register-based invalidation to complete
5b2be94 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): introduce IommuInfo structure to maintain Vt-d
1bcc26f 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
feat(vt-d): implement cahce invalidation
2b7a27d 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): update invalidate functions
2360367 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): invalidete cache when initializing
b6a5b5b 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): print some registers when invalidating cache
f878937 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): fix granuality
b37bf6e 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): check the fault status register
4a85e7d 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(idt): wait 0 for IRTPS first
b245438 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): invalidate cache when allocating and dealocating IRTE
d298fb8 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
Merge branch 'main' into fix_interrupt_remapping3
f776f69 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
fix(vt-d): minor fix
a9f8388 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano
minor fix
e1679c4 
Signed-off-by: Yuuki Takano <ytakanoster@gmail.com>
@ytakano ytakano changed the title fix(vt-d): invalidate cache when initializing fix(vt-d): invalidate cache when initializing and updating Jan 13, 2026
@ytakano ytakano requested a review from Koichi98 January 13, 2026 08:01
@ytakano ytakano marked this pull request as ready for review January 13, 2026 08:02
Koichi98
Koichi98 reviewed Jan 26, 2026
View reviewed changes
awkernel_lib/src/arch/x86_64/interrupt_remap.rs

// Check for completion
core::sync::atomic::fence(core::sync::atomic::Ordering::Acquire);
if status[0] != 0 {
Copy link
Contributor

@Koichi98 Koichi98 Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check for status[vtd_unit.index]?

awkernel_lib/src/arch/x86_64/interrupt_remap.rs
core::sync::atomic::fence(core::sync::atomic::Ordering::Release);

let mut node = MCSNode::new();
let vtd_units = IOMMU[self.segment_number].vtd_units.lock(&mut node);
Copy link
Contributor

@Koichi98 Koichi98 Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to add comment about the order of lock acquisition(interrupt_remapping->vtd_units).

awkernel_lib/src/arch/x86_64/interrupt_remap.rs
}
}
}

Copy link
Contributor

@Koichi98 Koichi98 Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add some kind of handling for timeout?

awkernel_lib/src/arch/x86_64/interrupt_remap.rs
array![_ => Mutex::new(None); 32];
static IOMMU: [IommuInfo; 32] = array![x =>
IommuInfo {
segment_number: x,
Copy link
Contributor

@Koichi98 Koichi98 Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like segment_number always being the same with the array index. Maybe we can delete segment_number field but do you want to keep it to make it explicit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Koichi98 Koichi98 Koichi98 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ytakano @Koichi98