S147 · SOURCE-BOUND GATE EVIDENCE
K2: Notification lifecycle ve post-admission revoke
Operations --test hedefi → focused test içindeki include_str!/#[path] bağı → kaynak kesiti Bu sayfa yalnız S147 kapısına aittir; komşu kapıların kaynakları bu kabulün içine katılmaz.
S147Focused kod testiOperations id exactsource SHA exacttest target exact
operation: k2-notification-lifecycle-revoke-partial
uygulama/model · focused test · Operations · 3 exact excerpt
sequence-bound=true · implementation-bound=true
01 · Testin bağlı olduğu uygulama/model kodu
Kapının yürüttüğü gerçek kaynak
tam Rust öğesiL446–L3588
kernel/src/task/scheduler.rs::notification_wait_timeout_or_park
impl Scheduler {
pub const fn new() -> Self {
Self {
ready_queue: BinaryHeap::new(),
current_task: None,
retired_task: None,
deferred_current_runtime_oom: None,
ticks_until_preempt: 0,
min_vruntime: 0,
ipc_blocked_tasks: spin::Mutex::new(Vec::new()),
}
}
/// Reap the task retired by an earlier context switch.
///
/// User roots remain paired with their non-zero ASID lease until this
/// later-stack reaper drops the complete page-table owner. The ASID is
/// returned only after those resources are gone.
fn reap_retired_task(&mut self) {
let Some(mut task) = self.retired_task.take() else {
return;
};
let deferred_match = self
.deferred_current_runtime_oom
.as_ref()
.filter(|carrier| carrier.witness.is_none())
.is_some_and(|carrier| {
task.id == carrier.binding.task_id
&& task.runtime_allocation_domain() == Some(carrier.binding.domain)
});
if deferred_match {
let mut carrier = self
.deferred_current_runtime_oom
.take()
.expect("deferred current OOM carrier disappeared");
let witness = reap_detached_runtime_oom_task(
task,
carrier.binding.domain,
carrier.ipc_lifecycle_closed,
);
carrier.witness = Some(witness);
self.deferred_current_runtime_oom = Some(carrier);
return;
}
if task.is_user && (task.asid == 0 || !task.address_space_quiesced) {
crate::kprintln!(
"[K1-LIFECYCLE] quarantine task #{} '{}' without complete address-space quiescence (asid={}); resources intentionally retained",
task.id,
task.name,
task.asid
);
core::mem::forget(task);
return;
}
crate::kprintln!(
"[K1-LIFECYCLE] reaping task #{} '{}' on a later task stack",
task.id,
task.name
);
unsafe {
task.release_owned_page_table_root();
}
task.owned_user_page_tables.take();
match crate::mm::address_space::reclaim_owned_user_frames(
&mut task.owned_user_frames,
&mut task.runtime_user_frames,
) {
Ok(Some(report)) if report.returned_to_baseline() => crate::kprintln!(
"[K1-RUNTIME-ELF-RECLAIM] task={} frames={} free={}->{} active_allocations={}->{} BASELINE=PASS",
task.id,
report.released_frames,
report.baseline_free_frames,
report.observed_free_frames,
report.baseline_active_allocations,
report.observed_active_allocations,
),
Ok(Some(report)) if report.exactly_reconciled() => crate::kprintln!(
"[K1-RUNTIME-ELF-RECLAIM] task={} frames={} free={}->{} active_allocations={}->{} RECONCILED=PASS BASELINE=CONCURRENT",
task.id,
report.released_frames,
report.reclaim_started_free_frames,
report.observed_free_frames,
report.reclaim_started_active_allocations,
report.observed_active_allocations,
),
Ok(Some(_)) => unreachable!("RuntimePmm reclaimer returned an unaudited delta"),
Ok(None) => {}
Err(error) => {
crate::kprintln!(
"[K1-RUNTIME-ELF-RECLAIM] task={} BASELINE=FAIL error={:?}; ASID and remaining resources quarantined",
task.id,
error
);
core::mem::forget(task);
return;
}
}
// Root/intermediate storage is unreachable and the prior TLBI is
// complete. Only now may a fresh address space acquire this ASID.
let retired_asid = task.asid;
if retired_asid != 0 {
if let Err(error) = crate::mm::address_space::free_asid(retired_asid) {
crate::kprintln!(
"[K1-ASID] final reaper release failed asid={} error={:?}; identifier remains quarantined",
retired_asid,
error
);
core::mem::forget(task);
return;
}
task.asid = 0;
}
// `Task` drop now releases its owned kernel/user stack allocations.
// Raw ELF user stacks have no `OwnedStackAllocation` and are not
// reconstructed as a Box.
drop(task);
}
/// CFS tarzı weight hesabı (düşük priority = daha yüksek weight = daha yavaş vruntime artışı)
fn weight_of(priority: u8) -> u64 {
match priority {
0 => 1024, // En yüksek öncelik
1 => 820,
2 => 655,
3 => 524,
4 => 419,
5 => 335,
6 => 268,
7 => 215,
_ => 128,
}
}
/// Scheduler içinde vruntime güncelleme (CFS-lite)
fn update_vruntime(&mut self, task: &mut Task, delta_exec: u64) {
let weight = Self::weight_of(task.priority);
let delta = delta_exec * 1024 / weight.max(1);
task.vruntime = task.vruntime.wrapping_add(delta);
// min_vruntime'ı güncelle (CFS'te scheduler bunu takip eder)
if task.vruntime < self.min_vruntime {
self.min_vruntime = task.vruntime;
}
}
/// Builds a complete task without exposing it through a scheduler
/// container. S354 and S355 use this with an owned S255 vruntime snapshot
/// so all fallible user/kernel preparation finishes before the respective
/// publication writer. The old unguarded `Scheduler::spawn` publication
/// path no longer exists.
fn build_unpublished_task(
name: &str,
kernel_entry: extern "C" fn() -> !,
stack_size: usize,
priority: u8,
time_slice: u32,
is_user: bool,
user_entry: Option<extern "C" fn() -> !>,
initial_user_arg0: u64,
initial_vruntime: u64,
) -> Result<(Task, JoinHandle), TaskSpawnError> {
const GUARD_SIZE: usize = 0x1000;
let validated_user_entry = if is_user {
Some(user_entry.ok_or(TaskSpawnError::MissingUserEntry)?)
} else {
None
};
let total_user_stack_alloc = if is_user {
Some(
stack_size
.checked_add(GUARD_SIZE)
.ok_or(TaskSpawnError::StackSizeOverflow)?,
)
} else {
None
};
// Task ids are authority principals. Reserve one before ASIDs, page
// tables, or stacks so permanent id exhaustion has no side effects.
// A later spawn failure may consume an id, but ids are never reused.
let id = TASK_ID_ALLOCATOR
.try_allocate()
.map_err(|_| TaskSpawnError::TaskIdExhausted)?;
// === M8.2+M7.4 fix (18-agent audit): page_table_root policy ===
// Kernel tasks share the live ROOT_PAGE_TABLE and never own or switch
// to a private root. The recorded value must nevertheless be its real
// address: framebuffer grants query this metadata and mutate the
// returned table. The old 0x4020_0000 placeholder points into the heap
// on current links and corrupts/walks allocator metadata as PTEs.
let kernel_root = unsafe {
let root = crate::arch::aarch64::mmu::get_kernel_root_table() as *mut _ as u64;
crate::mm::PhysAddr::new(root)
};
let (new_root, new_asid) = if is_user {
// Reserve the scarce identifier before any root or stack Box.
// Exhaustion is returned to the user-task caller without mutation.
let a = crate::mm::address_space::try_allocate_asid().map_err(TaskSpawnError::Asid)?;
let r = crate::mm::allocate_page_table_root();
(r, a)
} else {
(kernel_root, 0u16)
};
let kernel_stack_allocation = OwnedStackAllocation::new(stack_size);
let stack_top = unsafe { kernel_stack_allocation.top() };
let finished = Arc::new(AtomicBool::new(false));
// M5.5 (Audit #18) — EL0 stack + guard yalnız user task için gerekir.
// Kernel task'e ikinci, hiç kullanılmayan bir stack ayırmak hem heap'i
// tüketiyor hem de sahipliği kaybolan gereksiz bir allocation yaratıyordu.
let user_stack_allocation = total_user_stack_alloc.map(OwnedStackAllocation::new);
let (user_stack_ptr, usable_stack_bottom) = if let Some(owner) = &user_stack_allocation {
let user_stack_ptr = owner.base();
let usable_stack_bottom = unsafe { user_stack_ptr.add(GUARD_SIZE) };
// Guard sayfasının üstüne canary yaz (ileride overflow kontrolü).
unsafe {
core::ptr::write_volatile(
user_stack_ptr.add(GUARD_SIZE - 8) as *mut u64,
0xDEAD_BEEF_C0FFEE00,
);
}
(user_stack_ptr, usable_stack_bottom)
} else {
(core::ptr::null_mut(), core::ptr::null_mut())
};
// Sadece user task için per-task root mapping kur.
if is_user {
unsafe {
let root_table = crate::arch::aarch64::mmu::get_page_table_from_root(new_root);
// Düşük RAM + heap bölgesi (kernel tarafı NORMAL — kernel kodu için exec).
crate::arch::aarch64::mmu::map_range_4k_to_root(
root_table,
crate::mm::VirtAddr::new(0x4000_0000),
crate::mm::PhysAddr::new(0x4000_0000),
0x0800_0000,
crate::mm::paging::PageTableFlags::NORMAL,
);
// UART
crate::arch::aarch64::mmu::map_range_4k_to_root(
root_table,
crate::mm::VirtAddr::new(0x0900_0000),
crate::mm::PhysAddr::new(0x0900_0000),
0x0020_0000,
crate::mm::paging::PageTableFlags::DEVICE,
);
// GIC
crate::arch::aarch64::mmu::map_range_4k_to_root(
root_table,
crate::mm::VirtAddr::new(0x0800_0000),
crate::mm::PhysAddr::new(0x0800_0000),
0x0020_0000,
crate::mm::paging::PageTableFlags::DEVICE,
);
// M5.5+M7.4 fix: User stack USER_NORMAL override.
// Heap NORMAL kalır (kernel isolation), sadece user stack range'i
// USER_NORMAL ile EL0'a açılır. Guard page (alt 4K) MAP EDİLMEZ →
// translation fault overflow yakalar (Strategy A — Linux/seL4 patterni).
//
// KRİTİK: Box::into_raw u8 alignment'lı bir pointer döner (4K aligned değil).
// map_range_4k_to_root 4K page boundary bekler — unaligned adresleri
// atlar (page'leri map etmez) → user EL0 stack write'da perm fault.
// Bu yüzden adresi 4K'ya yuvarla (aşağı → start, yukarı → end).
let usable_stack_phys = usable_stack_bottom as u64;
let aligned_start = usable_stack_phys & !0xFFF; // round down
let aligned_end = (usable_stack_phys + stack_size as u64 + 0xFFF) & !0xFFF;
let aligned_size = aligned_end - aligned_start;
crate::arch::aarch64::mmu::map_range_4k_to_root(
root_table,
crate::mm::VirtAddr::new(aligned_start),
crate::mm::PhysAddr::new(aligned_start),
aligned_size,
crate::mm::paging::PageTableFlags::USER_NORMAL,
);
crate::kprintln!(
"[M5.5+M7.4] user '{}' root=0x{:x} asid={} stack USER_NORMAL [0x{:x}..0x{:x}] (size=0x{:x}, requested phys=0x{:x})",
name, new_root.as_u64(), new_asid, aligned_start, aligned_end, aligned_size, usable_stack_phys
);
}
}
let mut task = Task {
id,
name: String::from(name),
state: TaskState::Ready,
context: TaskContext::default(),
priority,
time_slice,
default_time_slice: time_slice,
vruntime: initial_vruntime,
finished: finished.clone(),
kernel_stack_allocation: Some(kernel_stack_allocation),
user_stack_allocation,
user_stack_bottom: usable_stack_bottom,
user_stack_size: if is_user { stack_size } else { 0 },
// M5.5: yalnız user task'te guard sayfası ayrılır.
stack_guard_page: if is_user { Some(user_stack_ptr) } else { None },
// M4.3
is_user,
user_sp: 0,
saved_user_elr: 0,
saved_user_spsr: 0,
saved_user_gprs: {
let mut registers = [0; 31];
if is_user {
registers[0] = initial_user_arg0;
}
registers
},
// Kernel task = live shared kernel root/asid=0; user task = unique owned root + ASID.
page_table_root: new_root,
owns_page_table_root: is_user,
owned_user_page_tables: None,
owned_user_frames: alloc::vec::Vec::new(),
runtime_user_frames: None,
asid: new_asid,
#[cfg(feature = "board-rpi5")]
g8l_current_task_owner: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerSlot::new(),
address_space_quiesced: !is_user,
// M5: Her task kendi CNode'unu alır (boş başlar)
cnode: crate::ui::capability::CNode::new(),
};
// Kernel context'i her zaman trampoline veya normal entry'ye işaret eder
// User task'ler için ilk girişe özel kurulum kullanıyoruz
unsafe {
if is_user {
task.context = TaskContext::new_for_user_first_entry(stack_top);
} else {
task.context = TaskContext::new(kernel_entry, stack_top);
}
}
// User task ise initial EL0 frame'i hemen hazırla
if let Some(entry) = validated_user_entry {
// Stack top = usable_bottom + size (guard hariç)
task.user_sp = usable_stack_bottom as u64 + stack_size as u64; // stack top (büyüme aşağı)
task.saved_user_elr = entry as u64;
task.saved_user_spsr = 0; // EL0t
// GPR'ler sıfır kalır, sadece PC ve SP önemli
}
let handle = JoinHandle {
finished,
task_id: id,
};
Ok((task, handle))
}
/// Build a kernel-task skeleton without publishing it in any scheduler
/// container. The raw-ELF path consumes this value, installs the complete
/// user address-space state, and performs exactly one final ready-queue
/// push. No CPU can therefore observe a placeholder entry/root/ASID.
fn build_unpublished_kernel_task(
name: &str,
kernel_entry: extern "C" fn() -> !,
stack_size: usize,
priority: u8,
time_slice: u32,
initial_vruntime: u64,
) -> Result<Box<Task>, &'static str> {
let id = TASK_ID_ALLOCATOR
.try_allocate()
.map_err(|_| "task id space exhausted")?;
let kernel_root = {
let root =
unsafe { crate::arch::aarch64::mmu::get_kernel_root_table() } as *mut _ as u64;
crate::mm::PhysAddr::new(root)
};
let kernel_stack_allocation = OwnedStackAllocation::new(stack_size);
let stack_top = unsafe { kernel_stack_allocation.top() };
let mut task = Box::new(Task {
id,
name: String::from(name),
state: TaskState::Ready,
context: TaskContext::default(),
priority,
time_slice,
default_time_slice: time_slice,
vruntime: initial_vruntime,
finished: Arc::new(AtomicBool::new(false)),
kernel_stack_allocation: Some(kernel_stack_allocation),
user_stack_allocation: None,
user_stack_bottom: core::ptr::null_mut(),
user_stack_size: 0,
stack_guard_page: None,
is_user: false,
user_sp: 0,
saved_user_elr: 0,
saved_user_spsr: 0,
saved_user_gprs: [0; 31],
page_table_root: kernel_root,
owns_page_table_root: false,
owned_user_page_tables: None,
owned_user_frames: alloc::vec::Vec::new(),
runtime_user_frames: None,
asid: 0,
#[cfg(feature = "board-rpi5")]
g8l_current_task_owner: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerSlot::new(),
address_space_quiesced: true,
cnode: crate::ui::capability::CNode::new(),
});
task.context = unsafe { TaskContext::new(kernel_entry, stack_top) };
Ok(task)
}
/// Mevcut task'i kuyruğa geri koyar ve en yüksek öncelikli (en düşük vruntime) task'i seçer.
///
/// NOT: Şu anda kullanılmıyor (`yield_now` aynı işi inline yapıyor).
/// İleride farklı bir API (örn. IRQ-driven preemption) lazım olursa bu
/// fonksiyon kullanılabilir.
#[allow(dead_code)]
pub unsafe fn schedule(&mut self) -> Option<(*mut TaskContext, *mut TaskContext)> {
if let Some(mut curr) = self.current_task.take() {
curr.state = TaskState::Ready;
if curr.time_slice == 0 {
curr.time_slice = curr.default_time_slice;
}
self.ready_queue.push(PriorityTask::new(curr));
}
// min_vruntime'ı güncelle (daha adil seçim için)
self.recalculate_min_vruntime();
if let Some(prio_next) = self.ready_queue.pop() {
let mut next = prio_next.task;
next.state = TaskState::Running;
let new_ctx = &mut next.context as *mut TaskContext;
self.current_task = Some(next);
self.ticks_until_preempt = self.current_task.as_ref().unwrap().time_slice;
Some((core::ptr::null_mut(), new_ctx))
} else {
None
}
}
/// Timer her tick'te çağrılır.
/// Time slice bitince preemption tetikler + vruntime ve aging uygular.
pub unsafe fn tick(&mut self) -> bool {
static TICK_COUNTER: AtomicU64 = AtomicU64::new(0);
let n = TICK_COUNTER.fetch_add(1, Ordering::Relaxed) + 1;
// Aging + min_vruntime + normalizasyon (her 50 tick'te bir)
if n % 50 == 0 {
self.apply_aging();
self.recalculate_min_vruntime();
self.normalize_vruntime();
}
if let Some(curr) = &mut self.current_task {
if curr.time_slice > 0 {
curr.time_slice -= 1;
}
// Vruntime'ı da tick bazında hafifçe artır
let weight = Self::weight_of(curr.priority);
curr.vruntime = curr.vruntime.wrapping_add(1 * 1024 / weight.max(1));
if curr.vruntime < self.min_vruntime {
self.min_vruntime = curr.vruntime;
}
if curr.time_slice == 0 {
return true; // preemption gerekli
}
}
false
}
/// Aging mekanizması:
/// Uzun süredir bekleyen task'lerin vruntime'ını azaltarak önlerine geçmelerini sağlar.
/// Bu sayede düşük öncelikli task'ler bile zamanla CPU alabilir.
fn apply_aging(&mut self) {
let mut temp = Vec::new();
while let Some(mut ptask) = self.ready_queue.pop() {
// min_vruntime'tan çok geride kalanlara daha fazla bonus veriyoruz
let bonus = if ptask.task.vruntime + 50 < self.min_vruntime {
12
} else {
6
};
ptask.task.vruntime = ptask.task.vruntime.saturating_sub(bonus);
temp.push(ptask);
}
for p in temp {
self.ready_queue.push(p);
}
}
/// Mevcut task'in context pointer'ını döner (güvenli kullanım için)
pub fn current_context_ptr(&self) -> Option<*mut TaskContext> {
self.current_task
.as_ref()
.map(|t| &t.context as *const _ as *mut TaskContext)
}
/// Mevcut çalışan task'in ID'sini döndürür.
pub fn current_task_id(&self) -> Option<u64> {
self.current_task.as_ref().map(|t| t.id)
}
/// S211 producer-side observation of the real scheduler current task.
/// The target adapter holds a local IRQ guard while calling this method;
/// cross-CPU exclusion against legacy scheduler accesses remains open.
#[cfg(feature = "board-rpi5")]
pub(crate) fn observe_g8l_current_task(
&self,
) -> Option<
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lObservedCurrentTask,
>{
self.current_task.as_ref().map(|task| {
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lObservedCurrentTask {
task_id: task.id,
running: task.state == TaskState::Running,
el0: task.is_user,
asid: task.asid,
root: task.page_table_root.as_u64(),
}
})
}
#[cfg(feature = "board-rpi5")]
pub(crate) fn commit_g8l_current_task_owner(
&mut self,
runtime: &crate::g8l_runtime_contract::G8lRuntimeAuthority,
authority: &crate::g8l_target_dispatch_scheduler_owner::G8lSchedulerOwnerAuthority,
bridge: &crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_guarded_ack::G8lSchedulerMutationProductionGuardedAckBridge<'_>,
) -> Result<
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitReceipt,
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitError,
>{
let task = self.current_task.as_mut().ok_or(
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitError::MissingCurrentTask,
)?;
let observed = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lObservedCurrentTask {
task_id: task.id,
running: task.state == TaskState::Running,
el0: task.is_user,
asid: task.asid,
root: task.page_table_root.as_u64(),
};
task.g8l_current_task_owner
.commit_from_guarded_bridge(runtime, authority, bridge, observed)
}
#[cfg(feature = "board-rpi5")]
pub(crate) fn rollback_g8l_current_task_owner(
&mut self,
receipt: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitReceipt,
) -> Result<
(),
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitError,
>{
let task = self.current_task.as_mut().ok_or(
crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitError::MissingCurrentTask,
)?;
if task.id != receipt.task_id {
return Err(crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_current_task::G8lSchedulerCurrentTaskOwnerCommitError::TaskMismatch);
}
task.g8l_current_task_owner.rollback_exact(receipt)
}
fn record_runtime_oom_binding(
task: &Task,
location: RuntimeOomTaskLocation,
domain: crate::mm::AllocationDomain,
expected_frames: usize,
binding: &mut Option<RuntimeOomTaskBinding>,
) -> Result<(), RuntimeOomTaskExecutionError> {
if task.runtime_allocation_domain() != Some(domain) {
return Ok(());
}
if binding.is_some() {
return Err(RuntimeOomTaskExecutionError::AmbiguousDomainBinding);
}
let Some(ledger) = task.runtime_user_frames.as_ref() else {
return Err(RuntimeOomTaskExecutionError::InvalidTaskBinding);
};
let expected_state = match location {
RuntimeOomTaskLocation::Current => task.state == TaskState::Running,
RuntimeOomTaskLocation::Ready => task.state == TaskState::Ready,
RuntimeOomTaskLocation::Blocked => matches!(
task.state,
TaskState::Blocked
| TaskState::BlockedOnIpc { .. }
| TaskState::BlockedOnNotification { .. }
),
RuntimeOomTaskLocation::Retired => task.state == TaskState::Dead,
};
if !task.is_user
|| task.asid == 0
|| task.address_space_quiesced
|| task.owns_page_table_root
|| task.owned_user_page_tables.is_none()
|| ledger.frame_count() != expected_frames
|| task.owned_user_frames.len() != expected_frames
|| !ledger.physical_frames_match(&task.owned_user_frames)
|| !expected_state
{
return Err(RuntimeOomTaskExecutionError::InvalidTaskBinding);
}
*binding = Some(RuntimeOomTaskBinding {
task_id: task.id,
domain,
location,
asid: task.asid,
frame_count: expected_frames,
});
Ok(())
}
/// Allocation-free audit of every persistent scheduler container. A
/// domain must bind one strict RuntimePmm ELF task and the ticket's frame
/// count must equal that task's complete leaf ledger.
fn audit_runtime_oom_task_binding(
&self,
preflight: crate::mm::RuntimeOomTeardownPreflight,
) -> Result<RuntimeOomTaskBinding, RuntimeOomTaskExecutionError> {
let expected_frames = usize::try_from(preflight.expected_reclaimable_frames)
.map_err(|_| RuntimeOomTaskExecutionError::InvalidTaskBinding)?;
let mut binding = None;
if let Some(task) = self.current_task.as_deref() {
Self::record_runtime_oom_binding(
task,
RuntimeOomTaskLocation::Current,
preflight.domain,
expected_frames,
&mut binding,
)?;
}
for priority_task in self.ready_queue.iter() {
Self::record_runtime_oom_binding(
&priority_task.task,
RuntimeOomTaskLocation::Ready,
preflight.domain,
expected_frames,
&mut binding,
)?;
}
{
let blocked = self.ipc_blocked_tasks.lock();
for task in blocked.iter() {
Self::record_runtime_oom_binding(
task,
RuntimeOomTaskLocation::Blocked,
preflight.domain,
expected_frames,
&mut binding,
)?;
}
}
if let Some(task) = self.retired_task.as_deref() {
Self::record_runtime_oom_binding(
task,
RuntimeOomTaskLocation::Retired,
preflight.domain,
expected_frames,
&mut binding,
)?;
}
binding.ok_or(RuntimeOomTaskExecutionError::DomainNotBound)
}
fn preflight_runtime_oom_task(
&self,
preflight: crate::mm::RuntimeOomTeardownPreflight,
) -> Result<RuntimeOomTaskBinding, RuntimeOomTaskExecutionError> {
let binding = self.audit_runtime_oom_task_binding(preflight)?;
match binding.location {
RuntimeOomTaskLocation::Current => {
Err(RuntimeOomTaskExecutionError::CurrentTaskRequiresDeferredExit)
}
RuntimeOomTaskLocation::Retired => {
Err(RuntimeOomTaskExecutionError::RetiredTaskQuarantined)
}
RuntimeOomTaskLocation::Ready | RuntimeOomTaskLocation::Blocked => Ok(binding),
}
}
/// Detach the exact preflighted non-current task without allocating. The
/// ready heap is rebuilt in-place from its existing Vec allocation.
fn take_runtime_oom_task(&mut self, binding: RuntimeOomTaskBinding) -> Option<Box<Task>> {
let mut ready = core::mem::take(&mut self.ready_queue).into_vec();
let ready_match = ready.iter().position(|priority_task| {
priority_task.task.id == binding.task_id
&& priority_task.task.runtime_allocation_domain() == Some(binding.domain)
});
let selected = ready_match.map(|index| ready.swap_remove(index).task);
self.ready_queue = BinaryHeap::from(ready);
if selected.is_some() {
return selected;
}
let mut blocked = self.ipc_blocked_tasks.lock();
let blocked_match = blocked.iter().position(|task| {
task.id == binding.task_id && task.runtime_allocation_domain() == Some(binding.domain)
});
blocked_match.map(|index| blocked.swap_remove(index))
}
pub(crate) fn capability_for_task(
&self,
task_id: u64,
cap_id: crate::ui::capability::CapId,
) -> Option<crate::ui::capability::Capability> {
if let Some(capability) = self
.current_task
.as_ref()
.filter(|task| task.id == task_id)
.and_then(|task| task.cnode.lookup_capability_by_id(cap_id))
.copied()
{
return Some(capability);
}
if let Some(capability) = self
.ipc_blocked_tasks
.lock()
.iter()
.find(|task| task.id == task_id)
.and_then(|task| task.cnode.lookup_capability_by_id(cap_id))
.copied()
{
return Some(capability);
}
if let Some(capability) = self
.ready_queue
.iter()
.find(|priority_task| priority_task.task.id == task_id)
.and_then(|priority_task| priority_task.task.cnode.lookup_capability_by_id(cap_id))
.copied()
{
return Some(capability);
}
self.retired_task
.as_ref()
.filter(|task| task.id == task_id)
.and_then(|task| task.cnode.lookup_capability_by_id(cap_id))
.copied()
}
pub(crate) fn notification_holder_count(
&self,
notification_id: crate::ui::capability::CapId,
) -> usize {
let holds = |task: &Task| {
task.cnode
.lookup_capability_by_id(notification_id)
.is_some_and(|capability| {
capability.kind == crate::ui::capability::CapabilityKind::Notification
})
};
let mut count = usize::from(self.current_task.as_deref().is_some_and(holds));
count += self
.ready_queue
.iter()
.filter(|task| holds(&task.task))
.count();
count += self
.ipc_blocked_tasks
.lock()
.iter()
.filter(|task| holds(task))
.count();
count += usize::from(self.retired_task.as_deref().is_some_and(holds));
count
}
fn notification_holder_for_task(
task: &Task,
notification_id: crate::ui::capability::CapId,
object_owner: u64,
) -> Result<Option<crate::ui::capability::Capability>, EndpointHolderPurgeError> {
use crate::ui::capability::{CapabilityKind, CapabilityRights};
let Some(capability) = task.cnode.lookup_capability_by_id(notification_id).copied() else {
return Ok(None);
};
let is_object_owner = object_owner != 0 && task.id == object_owner;
let expected_parent = if is_object_owner {
None
} else {
Some(notification_id)
};
let notification_rights = capability.rights.intersect(CapabilityRights::FULL);
if capability.id != notification_id
|| capability.owner != task.id
|| capability.kind != CapabilityKind::Notification
|| capability.parent != expected_parent
|| capability.rights == CapabilityRights::NONE
|| notification_rights != capability.rights
|| (is_object_owner
&& !capability
.rights
.contains(CapabilityRights::NOTIFICATION_REVOKE))
|| !task.cnode.can_revoke_capability_exact(&capability)
{
return Err(EndpointHolderPurgeError::MalformedAuthority);
}
Ok(Some(capability))
}
fn audit_notification_holder_task(
task: &Task,
notification_id: crate::ui::capability::CapId,
object_owner: u64,
container: TaskContainer,
summary: &mut EndpointHolderPurgeSummary,
) -> Result<(), EndpointHolderPurgeError> {
if Self::notification_holder_for_task(task, notification_id, object_owner)?.is_some() {
summary.record(container)?;
}
Ok(())
}
/// Generation-aware audit of every persistent notification holder. The
/// caller owns `IPC_TRANSACTION_LOCK`, so the matching purge observes the
/// same CNode graph.
pub(crate) fn preflight_notification_holder_purge(
&self,
notification_id: crate::ui::capability::CapId,
object_owner: u64,
) -> Result<EndpointHolderPurgeSummary, EndpointHolderPurgeError> {
let mut summary = EndpointHolderPurgeSummary::default();
if let Some(task) = self.current_task.as_deref() {
Self::audit_notification_holder_task(
task,
notification_id,
object_owner,
TaskContainer::Current,
&mut summary,
)?;
}
for task in self.ready_queue.iter() {
Self::audit_notification_holder_task(
&task.task,
notification_id,
object_owner,
TaskContainer::Ready,
&mut summary,
)?;
}
for task in self.ipc_blocked_tasks.lock().iter() {
Self::audit_notification_holder_task(
task,
notification_id,
object_owner,
TaskContainer::Blocked,
&mut summary,
)?;
}
if let Some(task) = self.retired_task.as_deref() {
Self::audit_notification_holder_task(
task,
notification_id,
object_owner,
TaskContainer::Retired,
&mut summary,
)?;
}
if object_owner != 0 && summary.total == 0 {
return Err(EndpointHolderPurgeError::MalformedAuthority);
}
Ok(summary)
}
fn purge_notification_holder_from_task(
task: &mut Task,
notification_id: crate::ui::capability::CapId,
object_owner: u64,
container: TaskContainer,
summary: &mut EndpointHolderPurgeSummary,
) {
let capability = Self::notification_holder_for_task(task, notification_id, object_owner)
.expect("preflighted notification holder became malformed");
let Some(capability) = capability else {
return;
};
assert_eq!(
task.cnode.revoke_capability_exact(&capability),
Some(capability),
"preflighted notification holder exact revoke failed"
);
summary
.record(container)
.expect("notification holder count overflowed during commit");
}
/// Allocation-free exact purge paired with
/// `preflight_notification_holder_purge`.
pub(crate) fn purge_notification_holders_exact(
&mut self,
notification_id: crate::ui::capability::CapId,
object_owner: u64,
) -> EndpointHolderPurgeSummary {
let expected = self
.preflight_notification_holder_purge(notification_id, object_owner)
.expect("notification holder graph changed after teardown preflight");
let mut removed = EndpointHolderPurgeSummary::default();
if let Some(task) = self.current_task.as_deref_mut() {
Self::purge_notification_holder_from_task(
task,
notification_id,
object_owner,
TaskContainer::Current,
&mut removed,
);
}
let ready = core::mem::take(&mut self.ready_queue);
let mut ready_tasks = ready.into_vec();
for task in ready_tasks.iter_mut() {
Self::purge_notification_holder_from_task(
&mut task.task,
notification_id,
object_owner,
TaskContainer::Ready,
&mut removed,
);
}
self.ready_queue = BinaryHeap::from(ready_tasks);
for task in self.ipc_blocked_tasks.lock().iter_mut() {
Self::purge_notification_holder_from_task(
task,
notification_id,
object_owner,
TaskContainer::Blocked,
&mut removed,
);
}
if let Some(task) = self.retired_task.as_deref_mut() {
Self::purge_notification_holder_from_task(
task,
notification_id,
object_owner,
TaskContainer::Retired,
&mut removed,
);
}
assert_eq!(
removed, expected,
"notification holder purge differed from exact preflight"
);
removed
}
fn endpoint_holder_for_task(
task: &Task,
endpoint_id: crate::ui::capability::CapId,
object_owner: u64,
) -> Result<Option<crate::ui::capability::Capability>, EndpointHolderPurgeError> {
use crate::ui::capability::{CapabilityKind, CapabilityRights};
let Some(capability) = task.cnode.lookup_capability_by_id(endpoint_id).copied() else {
return Ok(None);
};
let is_object_owner = object_owner != 0 && task.id == object_owner;
let expected_parent = if is_object_owner {
None
} else {
Some(endpoint_id)
};
let endpoint_rights = capability.rights.intersect(CapabilityRights::FULL);
if capability.id != endpoint_id
|| capability.owner != task.id
|| capability.kind != CapabilityKind::Endpoint
|| capability.parent != expected_parent
|| capability.rights == CapabilityRights::NONE
|| endpoint_rights != capability.rights
|| (is_object_owner
&& !capability
.rights
.contains(CapabilityRights::ENDPOINT_REVOKE))
|| !task.cnode.can_revoke_capability_exact(&capability)
{
return Err(EndpointHolderPurgeError::MalformedAuthority);
}
Ok(Some(capability))
}
fn audit_endpoint_holder_task(
task: &Task,
endpoint_id: crate::ui::capability::CapId,
object_owner: u64,
container: TaskContainer,
summary: &mut EndpointHolderPurgeSummary,
) -> Result<(), EndpointHolderPurgeError> {
if Self::endpoint_holder_for_task(task, endpoint_id, object_owner)?.is_some() {
summary.record(container)?;
}
Ok(())
}
/// Allocation-free, generation-aware audit of every scheduler-owned task
/// CNode that can outlive the current instruction. The caller holds the
/// global IPC transaction lock, so this snapshot remains stable through
/// the matching purge.
pub(crate) fn preflight_endpoint_holder_purge(
&self,
endpoint_id: crate::ui::capability::CapId,
object_owner: u64,
) -> Result<EndpointHolderPurgeSummary, EndpointHolderPurgeError> {
let mut summary = EndpointHolderPurgeSummary::default();
if let Some(task) = self.current_task.as_deref() {
Self::audit_endpoint_holder_task(
task,
endpoint_id,
object_owner,
TaskContainer::Current,
&mut summary,
)?;
}
for task in self.ready_queue.iter() {
Self::audit_endpoint_holder_task(
&task.task,
endpoint_id,
object_owner,
TaskContainer::Ready,
&mut summary,
)?;
}
for task in self.ipc_blocked_tasks.lock().iter() {
Self::audit_endpoint_holder_task(
task,
endpoint_id,
object_owner,
TaskContainer::Blocked,
&mut summary,
)?;
}
if let Some(task) = self.retired_task.as_deref() {
Self::audit_endpoint_holder_task(
task,
endpoint_id,
object_owner,
TaskContainer::Retired,
&mut summary,
)?;
}
if object_owner != 0 && summary.total == 0 {
return Err(EndpointHolderPurgeError::MalformedAuthority);
}
Ok(summary)
}
fn purge_endpoint_holder_from_task(
task: &mut Task,
endpoint_id: crate::ui::capability::CapId,
object_owner: u64,
container: TaskContainer,
summary: &mut EndpointHolderPurgeSummary,
) {
let capability = Self::endpoint_holder_for_task(task, endpoint_id, object_owner)
.expect("preflighted endpoint holder became malformed inside one IPC transaction");
let Some(capability) = capability else {
return;
};
assert_eq!(
task.cnode.revoke_capability_exact(&capability),
Some(capability),
"preflighted endpoint holder exact revoke failed"
);
summary
.record(container)
.expect("preflighted endpoint holder count overflowed during commit");
}
/// Commit the exact holder snapshot without allocating. BinaryHeap is
/// converted into and rebuilt from its existing Vec allocation in place;
/// CNode changes cannot affect heap ordering.
pub(crate) fn purge_endpoint_holders_exact(
&mut self,
endpoint_id: crate::ui::capability::CapId,
object_owner: u64,
) -> EndpointHolderPurgeSummary {
let expected = self
.preflight_endpoint_holder_purge(endpoint_id, object_owner)
.expect("endpoint holder graph changed after teardown preflight");
let mut removed = EndpointHolderPurgeSummary::default();
if let Some(task) = self.current_task.as_deref_mut() {
Self::purge_endpoint_holder_from_task(
task,
endpoint_id,
object_owner,
TaskContainer::Current,
&mut removed,
);
}
let ready = core::mem::take(&mut self.ready_queue);
let mut ready_tasks = ready.into_vec();
for task in ready_tasks.iter_mut() {
Self::purge_endpoint_holder_from_task(
&mut task.task,
endpoint_id,
object_owner,
TaskContainer::Ready,
&mut removed,
);
}
self.ready_queue = BinaryHeap::from(ready_tasks);
for task in self.ipc_blocked_tasks.lock().iter_mut() {
Self::purge_endpoint_holder_from_task(
task,
endpoint_id,
object_owner,
TaskContainer::Blocked,
&mut removed,
);
}
if let Some(task) = self.retired_task.as_deref_mut() {
Self::purge_endpoint_holder_from_task(
task,
endpoint_id,
object_owner,
TaskContainer::Retired,
&mut removed,
);
}
assert_eq!(
removed, expected,
"endpoint holder purge differed from its exact preflight"
);
removed
}
/// Belirli bir task'in page table root'unu döndürür (M4.4 büyük adım için).
pub fn get_task_page_table_root(&self, task_id: u64) -> Option<crate::mm::PhysAddr> {
// Basit lineer arama (demo için yeterli)
if let Some(ref curr) = self.current_task {
if curr.id == task_id {
return Some(curr.page_table_root);
}
}
for ptask in self.ready_queue.iter() {
if ptask.task.id == task_id {
return Some(ptask.task.page_table_root);
}
}
for task in self.ipc_blocked_tasks.lock().iter() {
if task.id == task_id {
return Some(task.page_table_root);
}
}
None
}
/// M5 — Belirli bir task'in CNode'una capability ekler (cross-task grant için kritik).
/// Hem current_task hem ready_queue içindeki task'leri tarar.
pub fn insert_cap_for_task(
&mut self,
task_id: u64,
cap: crate::ui::capability::Capability,
) -> Result<(usize, u64), &'static str> {
if let Some(ref mut curr) = self.current_task {
if curr.id == task_id {
return curr.cnode.insert(cap);
}
}
let mut temp: alloc::vec::Vec<PriorityTask> = alloc::vec::Vec::new();
let mut result: Result<(usize, u64), &'static str> = Err("Task not found");
while let Some(mut pt) = self.ready_queue.pop() {
if pt.task.id == task_id {
result = pt.task.cnode.insert(cap);
temp.push(pt);
break;
} else {
temp.push(pt);
}
}
for p in temp {
self.ready_queue.push(p);
}
result
}
/// Private id-only CNode removal. Public callers must pass through the
/// global wrapper, which rejects endpoint identities in favor of typed
/// grant/object APIs.
fn revoke_cap_for_task(
&mut self,
task_id: u64,
cap_id: crate::ui::capability::CapId,
) -> Option<usize> {
self.remove_cap_by_id(task_id, cap_id)
}
/// CNode seviyesinde id bazlı güçlü revoke (generation bump dahil)
fn remove_cap_by_id(
&mut self,
task_id: u64,
cap_id: crate::ui::capability::CapId,
) -> Option<usize> {
if let Some(ref mut curr) = self.current_task {
if curr.id == task_id {
if curr.cnode.revoke_capability(cap_id).is_some() {
return Some(0);
}
}
}
// Reply owners normally wait outside the ready queue. Search this
// fixed authority location before the heap fallback so reply consume
// and endpoint teardown do not allocate a temporary queue at OOM.
let mut blocked = self.ipc_blocked_tasks.lock();
for task in blocked.iter_mut() {
if task.id == task_id && task.cnode.revoke_capability(cap_id).is_some() {
return Some(0);
}
}
drop(blocked);
let mut temp: alloc::vec::Vec<PriorityTask> = alloc::vec::Vec::new();
let mut found: Option<usize> = None;
while let Some(mut pt) = self.ready_queue.pop() {
if pt.task.id == task_id {
if pt.task.cnode.revoke_capability(cap_id).is_some() {
found = Some(0);
}
temp.push(pt);
break;
} else {
temp.push(pt);
}
}
for p in temp {
self.ready_queue.push(p);
}
if found.is_some() {
return found;
}
None
}
/// Allocation-free removal of one complete CNode authority tuple. This
/// is used by typed endpoint/reply teardown where id-only revocation would
/// let a stale capability delete a newer generation.
pub(crate) fn revoke_cap_for_task_exact(
&mut self,
expected: &crate::ui::capability::Capability,
) -> bool {
let task_id = expected.owner;
if let Some(task) = self
.current_task
.as_deref_mut()
.filter(|task| task.id == task_id)
{
return task.cnode.revoke_capability_exact(expected).is_some();
}
{
let mut blocked = self.ipc_blocked_tasks.lock();
if let Some(task) = blocked.iter_mut().find(|task| task.id == task_id) {
return task.cnode.revoke_capability_exact(expected).is_some();
}
}
let ready = core::mem::take(&mut self.ready_queue);
let mut ready_tasks = ready.into_vec();
let removed = ready_tasks
.iter_mut()
.find(|task| task.task.id == task_id)
.map_or(false, |task| {
task.task.cnode.revoke_capability_exact(expected).is_some()
});
self.ready_queue = BinaryHeap::from(ready_tasks);
if removed {
return true;
}
self.retired_task
.as_deref_mut()
.filter(|task| task.id == task_id)
.map_or(false, |task| {
task.cnode.revoke_capability_exact(expected).is_some()
})
}
// =================================================================
// M6.2 — IPC Blocking / Waking (İskelet)
// =================================================================
fn save_current_ipc_context(&mut self, ctx: &ExceptionContext, user_sp: u64) {
if let Some(current) = &mut self.current_task {
current.is_user = true;
current.user_sp = user_sp;
current.saved_user_elr = ctx.elr_el1;
current.saved_user_spsr = ctx.spsr_el1;
current.saved_user_gprs[..30].copy_from_slice(&ctx.gpr);
current.saved_user_gprs[30] = ctx.lr;
}
}
/// Copy the wake result back into the still-live exception frame whose
/// kernel continuation was saved by `switch_after_ipc_park`.
fn restore_current_ipc_context(&self, ctx: &mut ExceptionContext) {
let current = self
.current_task
.as_ref()
.expect("IPC continuation resumed without its current task");
assert!(
current.is_user && current.state == TaskState::Running,
"IPC continuation resumed outside a running EL0 task"
);
ctx.gpr.copy_from_slice(¤t.saved_user_gprs[..30]);
ctx.lr = current.saved_user_gprs[30];
ctx.elr_el1 = current.saved_user_elr;
ctx.spsr_el1 = current.saved_user_spsr;
}
fn current_endpoint_authority_is_live(
&self,
task_id: u64,
endpoint_id: crate::ui::capability::CapId,
expected_generation: u64,
required_right: crate::ui::capability::CapabilityRights,
) -> bool {
self.current_task
.as_ref()
.filter(|task| task.id == task_id)
.and_then(|task| task.cnode.lookup_capability_by_id(endpoint_id))
.map_or(false, |capability| {
capability.generation == expected_generation
&& capability.owner == task_id
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability.rights.contains(required_right)
})
}
fn current_notification_authority_is_live(
&self,
task_id: u64,
notification_id: crate::ui::capability::CapId,
expected_generation: u64,
required_right: crate::ui::capability::CapabilityRights,
) -> bool {
self.current_task
.as_ref()
.filter(|task| task.id == task_id)
.and_then(|task| task.cnode.lookup_capability_by_id(notification_id))
.is_some_and(|capability| {
capability.generation == expected_generation
&& capability.owner == task_id
&& capability.kind == crate::ui::capability::CapabilityKind::Notification
&& capability.rights.contains(required_right)
})
}
fn write_ipc_delivery(
task: &mut Task,
message: crate::ui::capability::IpcMessage,
reply_cap_id: u64,
) {
task.saved_user_gprs[0] = crate::ipc::IpcError::Ok.as_u64();
task.saved_user_gprs[1] = message.label;
task.saved_user_gprs[2] = message.badge;
task.saved_user_gprs[3] = message.data[0];
task.saved_user_gprs[4] = message.data[1];
task.saved_user_gprs[5] = message.data[2];
task.saved_user_gprs[6] = message.data[3];
task.saved_user_gprs[7] = reply_cap_id;
}
/// A registered rendezvous receiver may be legacy (no deadline record) or
/// S145 timed. Any record for that task must be the exact receiver waiter;
/// another kind/generation is an authority-graph mismatch, not a hint to
/// deliver anyway.
fn exact_receive_deadline_for_waiter<const N: usize>(
deadlines: &crate::ipc_deadline::IpcCallDeadlineRegistry<N>,
receiver: crate::ipc_rendezvous::ReceiverWaiter,
endpoint_id: crate::ui::capability::CapId,
) -> Result<Option<crate::ipc_wait::WaitRecord>, crate::ipc::IpcError> {
let record = deadlines.task_snapshot(receiver.task_id());
if record.is_some_and(|record| {
record.kind().tag() != crate::ipc_wait::WaitKindTag::Receive
|| record.kind().object_id() != endpoint_id
|| record.kind().object_generation() != receiver.cap_generation()
|| record.kind().reply_cap_id().is_some()
}) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
Ok(record)
}
fn retire_receive_deadline_after_delivery<const N: usize>(
deadlines: &mut crate::ipc_deadline::IpcCallDeadlineRegistry<N>,
record: Option<crate::ipc_wait::WaitRecord>,
) {
if let Some(record) = record {
deadlines
.complete_delivery_exact(record)
.expect("CALL delivered but exact RECV deadline did not retire");
}
}
fn switch_after_ipc_park_with_membership_handoff<BeforeSwitch, AfterResume>(
&mut self,
parked_context: *mut TaskContext,
before_switch: BeforeSwitch,
after_resume: AfterResume,
) where
BeforeSwitch: FnOnce(),
AfterResume: FnOnce(&mut Self),
{
let Some(priority_task) = self.ready_queue.pop() else {
panic!("transactional IPC parked the current task without a runnable successor");
};
let mut next = priority_task.task;
next.state = TaskState::Running;
let new_context = &mut next.context as *mut TaskContext;
let next_slice = next.default_time_slice;
// IPC blocking is a real scheduler transition, so it must install the
// next task's TTBR0/ASID just like yield_now does. The context saved by
// context_switch belongs to the parked Box<Task>; a stack-local dummy
// loses the live syscall continuation and can resume a stale frame.
unsafe {
prepare_task_for_context_switch(&next);
}
self.current_task = Some(next);
self.ticks_until_preempt = next_slice;
// A gate lease belongs to the task executing this continuation. It
// must not remain live while the selected task runs. S373 uses these
// callbacks to release immediately before the machine switch and to
// rejoin before reading the wake payload when this continuation later
// resumes. Other IPC paths retain the legacy no-op wrapper below.
before_switch();
unsafe {
crate::arch::aarch64::context_switch(parked_context, new_context);
}
after_resume(self);
}
fn switch_after_ipc_park(&mut self, parked_context: *mut TaskContext) {
self.switch_after_ipc_park_with_membership_handoff(parked_context, || {}, |_| {});
}
/// Atomically publishes a CALL and parks its caller under one lock order:
/// IPC transaction -> global endpoint table -> global blocked-task set.
/// The caller is in the reply wait set before a receiver becomes runnable.
pub fn ipc_call_commit_and_park(
&mut self,
ctx: &mut ExceptionContext,
user_sp: u64,
target_endpoint: crate::ui::capability::CapId,
expected_generation: u64,
reply_cap_id: crate::ui::capability::CapId,
message: crate::ui::capability::IpcMessage,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s374_irq_guard: crate::arch::aarch64::IrqGuard,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s374_writer_access: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s374_el0_ipc_call_writer_guard_integration::G8lS374ProductionSchedulerWriterAccess,
) -> Result<(), crate::ipc::IpcError> {
use crate::ipc_rendezvous::{CallError, CallOutcome, FinishOutcome};
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let caller_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_endpoint_authority_is_live(
caller_task,
target_endpoint,
expected_generation,
crate::ui::capability::CapabilityRights::ENDPOINT_SEND,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let target_index = endpoints
.iter()
.position(|endpoint| endpoint.id == target_endpoint && !endpoint.is_reply_cap)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let reply_is_linked = endpoints.iter().any(|endpoint| {
endpoint.id == reply_cap_id
&& endpoint.is_reply_cap
&& endpoint.owner == caller_task
&& endpoint.reply_target == Some(target_endpoint)
});
if !reply_is_linked {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiting_receiver = endpoints[target_index].rendezvous.waiting_receiver();
if waiting_receiver.is_none() && self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
if waiting_receiver.is_some() {
self.ready_queue
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
}
self.save_current_ipc_context(ctx, user_sp);
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
let receiver_deadline = if let Some(receiver) = waiting_receiver {
let receiver_task = blocked.iter().find(|task| {
task.id == receiver.task_id()
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
} if endpoint_id == target_endpoint
)
});
let receiver_authority_is_live = receiver_task
.and_then(|task| task.cnode.lookup_capability_by_id(target_endpoint))
.map_or(false, |capability| {
capability.generation == receiver.cap_generation()
&& capability.owner == receiver.task_id()
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability
.rights
.contains(crate::ui::capability::CapabilityRights::ENDPOINT_RECV)
});
if receiver.wait_token() != target_endpoint || !receiver_authority_is_live {
return Err(crate::ipc::IpcError::InvalidCapability);
}
Self::exact_receive_deadline_for_waiter(&deadlines, receiver, target_endpoint)?
} else {
None
};
let outcome = endpoints[target_index]
.rendezvous
.call(caller_task, reply_cap_id, message)
.map_err(|error| match error {
CallError::QueueFull | CallError::ReplyTableFull => crate::ipc::IpcError::QueueFull,
_ => crate::ipc::IpcError::InvalidCapability,
})?;
match endpoints[target_index]
.rendezvous
.finish_call_park(caller_task, reply_cap_id)
{
Ok(FinishOutcome::Park) => {}
Ok(FinishOutcome::Ready(_)) | Ok(FinishOutcome::Cancelled) | Err(_) => {
// CALL, finish and scheduler publication are serialized by
// IPC_TRANSACTION_LOCK. No REPLY/revoke transition can run
// between `call` and this point, so any non-Park result means
// the model and scheduler have already diverged. Returning a
// recoverable syscall error here would strand a published
// request/reply record; fail-stop before making it worse.
panic!("CALL rendezvous changed inside one IPC transaction")
}
}
let mut caller = self
.current_task
.take()
.expect("preflighted CALL current task disappeared");
caller.state = TaskState::BlockedOnIpc {
endpoint_id: reply_cap_id,
is_call: true,
};
// Moving the Box through the blocked/ready/current containers does not
// move its Task pointee, so this exact SAVE target stays valid until
// the caller is scheduled and resumes this syscall continuation.
let caller_context = &mut caller.context as *mut TaskContext;
blocked.push(caller);
match outcome {
CallOutcome::Queued => {}
CallOutcome::Deliver {
receiver_task,
receiver_wait_token,
request,
..
} => {
if receiver_wait_token != target_endpoint {
panic!("rendezvous receiver wait-token changed during CALL commit");
}
let receiver_position = blocked
.iter()
.position(|task| {
task.id == receiver_task
&& matches!(
task.state,
TaskState::BlockedOnIpc { endpoint_id, .. }
if endpoint_id == target_endpoint
)
})
.expect("preflighted rendezvous receiver disappeared");
let mut receiver = blocked.remove(receiver_position);
Self::retire_receive_deadline_after_delivery(&mut deadlines, receiver_deadline);
Self::write_ipc_delivery(&mut receiver, request, reply_cap_id);
receiver.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(receiver));
}
}
drop(blocked);
drop(endpoints);
drop(deadlines);
drop(transaction);
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
self.switch_after_ipc_park_with_membership_handoff(
caller_context,
|| {
drop(s374_writer_access);
drop(s374_irq_guard);
},
|scheduler| {
let s374_resume_irq_guard = crate::arch::aarch64::IrqGuard::new();
let s374_resume_writer_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s374_el0_ipc_call_writer_guard_integration::acquire_s374_production_scheduler_writer_access()
.unwrap_or_else(|error| {
panic!(
"S374 resumed normal EL0 IPC-call scheduler writer guard failed closed: {:?}",
error
)
});
scheduler.restore_current_ipc_context(ctx);
drop(s374_resume_writer_access);
drop(s374_resume_irq_guard);
},
);
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
{
self.switch_after_ipc_park(caller_context);
self.restore_current_ipc_context(ctx);
}
Ok(())
}
/// S144 deadline-bearing CALL. This is deliberately a distinct ABI path:
/// legacy SYS_IPC_CALL never interprets an unspecified x6 register. The
/// fixed wait record is armed under `IPC_TRANSACTION_LOCK` before the
/// rendezvous request becomes visible, and every fallible allocation or
/// authority check precedes that publication point.
#[allow(clippy::too_many_arguments)]
pub fn ipc_call_timeout_commit_and_park(
&mut self,
ctx: &mut ExceptionContext,
user_sp: u64,
target_endpoint: crate::ui::capability::CapId,
expected_generation: u64,
reply_cap_id: crate::ui::capability::CapId,
reply_generation: u64,
message: crate::ui::capability::IpcMessage,
now_tick: u64,
timeout_ticks: u64,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s379_irq_guard: crate::arch::aarch64::IrqGuard,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s379_writer_access: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s379_el0_ipc_call_timeout_writer_guard_integration::G8lS379ProductionSchedulerWriterAccess,
) -> Result<(), crate::ipc::IpcError> {
use crate::ipc_deadline::IpcCallDeadlineRegistryError;
use crate::ipc_rendezvous::{CallError, CallOutcome, FinishOutcome};
use crate::ipc_wait::WaitError;
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let caller_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_endpoint_authority_is_live(
caller_task,
target_endpoint,
expected_generation,
crate::ui::capability::CapabilityRights::ENDPOINT_SEND,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let reply_authority_is_live = self
.current_task
.as_ref()
.and_then(|task| task.cnode.lookup_capability_by_id(reply_cap_id))
.is_some_and(|capability| {
capability.id == reply_cap_id
&& capability.owner == caller_task
&& capability.generation == reply_generation
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability.parent.is_none()
});
if !reply_authority_is_live {
return Err(crate::ipc::IpcError::InvalidCapability);
}
// Global lock order for every deadline race is:
// IPC transaction -> deadline registry -> endpoint/provenance ->
// scheduler blocked set. Timer and REPLY follow the same order.
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let target_index = endpoints
.iter()
.position(|endpoint| endpoint.id == target_endpoint && !endpoint.is_reply_cap)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let reply_is_linked = endpoints.iter().any(|endpoint| {
endpoint.id == reply_cap_id
&& endpoint.is_reply_cap
&& endpoint.owner == caller_task
&& endpoint.reply_target == Some(target_endpoint)
});
if !reply_is_linked {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiting_receiver = endpoints[target_index].rendezvous.waiting_receiver();
if waiting_receiver.is_none() && self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
if waiting_receiver.is_some() {
self.ready_queue
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
}
self.save_current_ipc_context(ctx, user_sp);
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
let receiver_deadline = if let Some(receiver) = waiting_receiver {
let receiver_task = blocked.iter().find(|task| {
task.id == receiver.task_id()
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
} if endpoint_id == target_endpoint
)
});
let receiver_authority_is_live = receiver_task
.and_then(|task| task.cnode.lookup_capability_by_id(target_endpoint))
.is_some_and(|capability| {
capability.generation == receiver.cap_generation()
&& capability.owner == receiver.task_id()
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability
.rights
.contains(crate::ui::capability::CapabilityRights::ENDPOINT_RECV)
});
if receiver.wait_token() != target_endpoint || !receiver_authority_is_live {
return Err(crate::ipc::IpcError::InvalidCapability);
}
Self::exact_receive_deadline_for_waiter(&deadlines, receiver, target_endpoint)?
} else {
None
};
let deadline_record = deadlines
.register_call(
caller_task,
target_endpoint,
expected_generation,
reply_cap_id,
reply_generation,
now_tick,
timeout_ticks,
)
.map_err(|error| match error {
IpcCallDeadlineRegistryError::Wait(WaitError::TableFull) => {
crate::ipc::IpcError::QueueFull
}
IpcCallDeadlineRegistryError::Wait(
WaitError::DeadlineNotFuture | WaitError::DeadlineTooFar,
)
| IpcCallDeadlineRegistryError::WaitEpochExhausted => {
crate::ipc::IpcError::InvalidDeadline
}
IpcCallDeadlineRegistryError::Wait(_) => crate::ipc::IpcError::InvalidCapability,
})?;
let outcome =
match endpoints[target_index]
.rendezvous
.call(caller_task, reply_cap_id, message)
{
Ok(outcome) => outcome,
Err(error) => {
deadlines
.cancel_exact(deadline_record)
.expect("failed CALL publication must roll back its exact deadline");
return Err(match error {
CallError::QueueFull | CallError::ReplyTableFull => {
crate::ipc::IpcError::QueueFull
}
_ => crate::ipc::IpcError::InvalidCapability,
});
}
};
match endpoints[target_index]
.rendezvous
.finish_call_park(caller_task, reply_cap_id)
{
Ok(FinishOutcome::Park) => {}
Ok(FinishOutcome::Ready(_)) | Ok(FinishOutcome::Cancelled) | Err(_) => {
panic!("deadline CALL rendezvous changed inside one IPC transaction")
}
}
let mut caller = self
.current_task
.take()
.expect("preflighted deadline CALL current task disappeared");
caller.state = TaskState::BlockedOnIpc {
endpoint_id: reply_cap_id,
is_call: true,
};
let caller_context = &mut caller.context as *mut TaskContext;
blocked.push(caller);
match outcome {
CallOutcome::Queued => {}
CallOutcome::Deliver {
receiver_task,
receiver_wait_token,
request,
..
} => {
if receiver_wait_token != target_endpoint {
panic!("deadline CALL receiver wait-token changed during commit");
}
let receiver_position = blocked
.iter()
.position(|task| {
task.id == receiver_task
&& matches!(
task.state,
TaskState::BlockedOnIpc { endpoint_id, .. }
if endpoint_id == target_endpoint
)
})
.expect("preflighted deadline receiver disappeared");
let mut receiver = blocked.remove(receiver_position);
Self::retire_receive_deadline_after_delivery(&mut deadlines, receiver_deadline);
Self::write_ipc_delivery(&mut receiver, request, reply_cap_id);
receiver.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(receiver));
}
}
drop(blocked);
drop(endpoints);
drop(deadlines);
drop(transaction);
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
self.switch_after_ipc_park_with_membership_handoff(
caller_context,
|| {
drop(s379_writer_access);
drop(s379_irq_guard);
},
|scheduler| {
let s379_resume_irq_guard = crate::arch::aarch64::IrqGuard::new();
let s379_resume_writer_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s379_el0_ipc_call_timeout_writer_guard_integration::acquire_s379_production_scheduler_writer_access()
.unwrap_or_else(|error| {
panic!(
"S379 resumed EL0 IPC call-timeout scheduler writer guard failed closed: {:?}",
error
)
});
scheduler.restore_current_ipc_context(ctx);
drop(s379_resume_writer_access);
drop(s379_resume_irq_guard);
},
);
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
{
self.switch_after_ipc_park(caller_context);
self.restore_current_ipc_context(ctx);
}
Ok(())
}
/// Publish one synchronous kernel-supervisor request through the same
/// bounded endpoint rendezvous used by EL0 CALL, then block the current
/// kernel task until an ordinary EL0 REPLY wakes it. This is not an
/// ambient kernel message queue: the current task must own a live SEND
/// capability, the reply object is one-shot, and every transition remains
/// under the production IPC transaction/registry/scheduler lock order.
pub fn ipc_kernel_call_and_wait(
&mut self,
target_endpoint: crate::ui::capability::CapId,
expected_generation: u64,
reply_cap_id: crate::ui::capability::CapId,
message: crate::ui::capability::IpcMessage,
) -> Result<crate::ui::capability::IpcMessage, crate::ipc::IpcError> {
use crate::ipc_rendezvous::{CallError, CallOutcome, FinishOutcome};
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let caller_task = self
.current_task
.as_ref()
.filter(|task| !task.is_user)
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_endpoint_authority_is_live(
caller_task,
target_endpoint,
expected_generation,
crate::ui::capability::CapabilityRights::ENDPOINT_SEND,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let target_index = endpoints
.iter()
.position(|endpoint| endpoint.id == target_endpoint && !endpoint.is_reply_cap)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let reply_is_linked = endpoints.iter().any(|endpoint| {
endpoint.id == reply_cap_id
&& endpoint.is_reply_cap
&& endpoint.owner == caller_task
&& endpoint.reply_target == Some(target_endpoint)
});
if !reply_is_linked {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiting_receiver = endpoints[target_index].rendezvous.waiting_receiver();
if waiting_receiver.is_none() && self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
if waiting_receiver.is_some() {
self.ready_queue
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
}
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
let receiver_deadline = if let Some(receiver) = waiting_receiver {
let receiver_authority_is_live = blocked
.iter()
.find(|task| {
task.id == receiver.task_id()
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
} if endpoint_id == target_endpoint
)
})
.and_then(|task| task.cnode.lookup_capability_by_id(target_endpoint))
.map_or(false, |capability| {
capability.generation == receiver.cap_generation()
&& capability.owner == receiver.task_id()
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability
.rights
.contains(crate::ui::capability::CapabilityRights::ENDPOINT_RECV)
});
if receiver.wait_token() != target_endpoint || !receiver_authority_is_live {
return Err(crate::ipc::IpcError::InvalidCapability);
}
Self::exact_receive_deadline_for_waiter(&deadlines, receiver, target_endpoint)?
} else {
None
};
let outcome = endpoints[target_index]
.rendezvous
.call(caller_task, reply_cap_id, message)
.map_err(|error| match error {
CallError::QueueFull | CallError::ReplyTableFull => crate::ipc::IpcError::QueueFull,
_ => crate::ipc::IpcError::InvalidCapability,
})?;
if !matches!(
endpoints[target_index]
.rendezvous
.finish_call_park(caller_task, reply_cap_id),
Ok(FinishOutcome::Park)
) {
panic!("kernel CALL rendezvous changed inside one IPC transaction");
}
let mut caller = self
.current_task
.take()
.expect("preflighted kernel CALL current task disappeared");
caller.state = TaskState::BlockedOnIpc {
endpoint_id: reply_cap_id,
is_call: true,
};
let caller_context = &mut caller.context as *mut TaskContext;
blocked.push(caller);
match outcome {
CallOutcome::Queued => {}
CallOutcome::Deliver {
receiver_task,
receiver_wait_token,
request,
..
} => {
if receiver_wait_token != target_endpoint {
panic!("kernel CALL receiver wait-token changed during commit");
}
let receiver_position = blocked
.iter()
.position(|task| {
task.id == receiver_task
&& matches!(
task.state,
TaskState::BlockedOnIpc { endpoint_id, .. }
if endpoint_id == target_endpoint
)
})
.expect("preflighted kernel CALL receiver disappeared");
let mut receiver = blocked.remove(receiver_position);
Self::retire_receive_deadline_after_delivery(&mut deadlines, receiver_deadline);
Self::write_ipc_delivery(&mut receiver, request, reply_cap_id);
receiver.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(receiver));
}
}
drop(blocked);
drop(endpoints);
drop(deadlines);
drop(transaction);
self.switch_after_ipc_park(caller_context);
let caller = self
.current_task
.as_ref()
.filter(|task| task.id == caller_task && !task.is_user)
.expect("kernel CALL continuation resumed under another task");
if caller.saved_user_gprs[0] != crate::ipc::IpcError::Ok.as_u64()
|| caller.saved_user_gprs[7] != 0
{
return Err(crate::ipc::IpcError::InvalidCapability);
}
Ok(crate::ui::capability::IpcMessage {
label: caller.saved_user_gprs[1],
badge: caller.saved_user_gprs[2],
data: [
caller.saved_user_gprs[3],
caller.saved_user_gprs[4],
caller.saved_user_gprs[5],
caller.saved_user_gprs[6],
],
})
}
/// Atomically performs RECV's FIFO-pop-or-waiter-registration decision.
/// `Some` is an immediate queued delivery; `None` means the task was
/// parked and a context switch was initiated.
pub fn ipc_recv_or_park(
&mut self,
ctx: &mut ExceptionContext,
user_sp: u64,
endpoint_id: crate::ui::capability::CapId,
expected_generation: u64,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s373_irq_guard: crate::arch::aarch64::IrqGuard,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s373_writer_access: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s373_el0_ipc_receive_writer_guard_integration::G8lS373ProductionSchedulerWriterAccess,
) -> Result<Option<crate::ui::capability::IpcEnvelope>, crate::ipc::IpcError> {
use crate::ipc_rendezvous::{ReceiverWaiter, RecvOutcome};
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let receiver_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_endpoint_authority_is_live(
receiver_task,
endpoint_id,
expected_generation,
crate::ui::capability::CapabilityRights::ENDPOINT_RECV,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiter =
ReceiverWaiter::new_with_generation(receiver_task, endpoint_id, expected_generation)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let target = endpoints
.iter_mut()
.find(|endpoint| endpoint.id == endpoint_id && !endpoint.is_reply_cap)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let will_park = target.rendezvous.queued_len() == 0;
if will_park && self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
if will_park {
self.save_current_ipc_context(ctx, user_sp);
}
// Only the waiter-registration branch needs blocked-set capacity.
// Reserving on an immediate FIFO delivery could spuriously reject a
// receive under allocator pressure even though it does not allocate.
let mut blocked = if will_park {
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
Some(blocked)
} else {
None
};
match target
.rendezvous
.recv(waiter)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?
{
RecvOutcome::Deliver {
request,
reply_token,
..
} => {
drop(blocked);
drop(endpoints);
drop(transaction);
Ok(Some(crate::ui::capability::IpcEnvelope {
message: request,
reply_cap_id: reply_token,
}))
}
RecvOutcome::Registered => {
let mut blocked_guard = blocked
.take()
.expect("RECV registered without preflighted blocked capacity");
let mut receiver = self
.current_task
.take()
.expect("preflighted RECV current task disappeared");
receiver.state = TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
};
let receiver_context = &mut receiver.context as *mut TaskContext;
blocked_guard.push(receiver);
drop(blocked_guard);
drop(blocked);
drop(endpoints);
drop(transaction);
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
self.switch_after_ipc_park_with_membership_handoff(
receiver_context,
|| {
drop(s373_writer_access);
drop(s373_irq_guard);
},
|scheduler| {
let s373_resume_irq_guard = crate::arch::aarch64::IrqGuard::new();
let s373_resume_writer_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s373_el0_ipc_receive_writer_guard_integration::acquire_s373_production_scheduler_writer_access()
.unwrap_or_else(|error| {
panic!(
"S373 resumed EL0 IPC-receive scheduler writer guard failed closed: {:?}",
error
)
});
scheduler.restore_current_ipc_context(ctx);
drop(s373_resume_writer_access);
drop(s373_resume_irq_guard);
},
);
#[cfg(not(all(
target_arch = "aarch64",
target_os = "none",
feature = "board-rpi5"
)))]
{
self.switch_after_ipc_park(receiver_context);
self.restore_current_ipc_context(ctx);
}
Ok(None)
}
}
}
/// S145 opt-in deadline-bearing RECV. The legacy syscall above remains
/// byte-for-byte policy compatible and never interprets x6. Only an empty
/// endpoint arms a wait record; a queued FIFO message is returned with
/// `IMMEDIATE_DELIVERY_DEADLINE=UNARMED`.
pub fn ipc_recv_timeout_or_park(
&mut self,
ctx: &mut ExceptionContext,
user_sp: u64,
endpoint_id: crate::ui::capability::CapId,
expected_generation: u64,
now_tick: u64,
timeout_ticks: u64,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s378_irq_guard: crate::arch::aarch64::IrqGuard,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s378_writer_access: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s378_el0_ipc_receive_timeout_writer_guard_integration::G8lS378ProductionSchedulerWriterAccess,
) -> Result<Option<crate::ui::capability::IpcEnvelope>, crate::ipc::IpcError> {
use crate::ipc_deadline::IpcCallDeadlineRegistryError;
use crate::ipc_rendezvous::{ReceiverWaiter, RecvOutcome};
use crate::ipc_wait::WaitError;
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let receiver_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_endpoint_authority_is_live(
receiver_task,
endpoint_id,
expected_generation,
crate::ui::capability::CapabilityRights::ENDPOINT_RECV,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiter =
ReceiverWaiter::new_with_generation(receiver_task, endpoint_id, expected_generation)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
// Shared CALL/RECV arbitration order: transaction -> deadline table ->
// endpoint -> blocked scheduler set.
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let target_index = endpoints
.iter()
.position(|endpoint| endpoint.id == endpoint_id && !endpoint.is_reply_cap)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let will_park = endpoints[target_index].rendezvous.queued_len() == 0;
if will_park && self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
let mut blocked = if will_park {
self.save_current_ipc_context(ctx, user_sp);
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
Some(blocked)
} else {
None
};
if !will_park {
// S145 acceptance marker: IMMEDIATE_DELIVERY_DEADLINE=UNARMED.
let outcome = endpoints[target_index]
.rendezvous
.recv(waiter)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
let RecvOutcome::Deliver {
request,
reply_token,
..
} = outcome
else {
panic!("non-empty endpoint registered a timed RECV waiter")
};
drop(blocked);
drop(endpoints);
drop(deadlines);
drop(transaction);
return Ok(Some(crate::ui::capability::IpcEnvelope {
message: request,
reply_cap_id: reply_token,
}));
}
let deadline_record = deadlines
.register_receive(
receiver_task,
endpoint_id,
expected_generation,
now_tick,
timeout_ticks,
)
.map_err(|error| match error {
IpcCallDeadlineRegistryError::Wait(WaitError::TableFull) => {
crate::ipc::IpcError::QueueFull
}
IpcCallDeadlineRegistryError::Wait(
WaitError::DeadlineNotFuture | WaitError::DeadlineTooFar,
)
| IpcCallDeadlineRegistryError::WaitEpochExhausted => {
crate::ipc::IpcError::InvalidDeadline
}
IpcCallDeadlineRegistryError::Wait(_) => crate::ipc::IpcError::InvalidCapability,
})?;
let outcome = endpoints[target_index].rendezvous.recv(waiter);
match outcome {
Ok(RecvOutcome::Registered) => {}
Ok(RecvOutcome::Deliver { .. }) | Err(_) => {
deadlines
.cancel_exact(deadline_record)
.expect("failed RECV publication must roll back its exact deadline");
return Err(crate::ipc::IpcError::InvalidCapability);
}
}
let mut blocked_guard = blocked
.take()
.expect("timed RECV registered without blocked capacity");
let mut receiver = self
.current_task
.take()
.expect("preflighted timed RECV current task disappeared");
receiver.state = TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
};
let receiver_context = &mut receiver.context as *mut TaskContext;
blocked_guard.push(receiver);
drop(blocked_guard);
drop(blocked);
drop(endpoints);
drop(deadlines);
drop(transaction);
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
self.switch_after_ipc_park_with_membership_handoff(
receiver_context,
|| {
drop(s378_writer_access);
drop(s378_irq_guard);
},
|scheduler| {
let s378_resume_irq_guard = crate::arch::aarch64::IrqGuard::new();
let s378_resume_writer_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s378_el0_ipc_receive_timeout_writer_guard_integration::acquire_s378_production_scheduler_writer_access()
.unwrap_or_else(|error| {
panic!(
"S378 resumed EL0 IPC receive-timeout scheduler writer guard failed closed: {:?}",
error
)
});
scheduler.restore_current_ipc_context(ctx);
drop(s378_resume_writer_access);
drop(s378_resume_irq_guard);
},
);
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
{
self.switch_after_ipc_park(receiver_context);
self.restore_current_ipc_context(ctx);
}
Ok(None)
}
/// S146 timed notification wait. Matching bits already pending are
/// consumed immediately without a deadline record. Otherwise the shared
/// deadline is published before the exact object waiter.
pub fn notification_wait_timeout_or_park(
&mut self,
ctx: &mut ExceptionContext,
user_sp: u64,
notification_id: crate::ui::capability::CapId,
expected_generation: u64,
mask: u64,
now_tick: u64,
timeout_ticks: u64,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s377_irq_guard: crate::arch::aarch64::IrqGuard,
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
s377_writer_access: crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s377_el0_notification_wait_timeout_writer_guard_integration::G8lS377ProductionSchedulerWriterAccess,
) -> Result<Option<u64>, crate::ipc::IpcError> {
use crate::ipc_deadline::IpcCallDeadlineRegistryError;
use crate::ipc_notification::{NotificationWaitOutcome, NotificationWaiter};
use crate::ipc_wait::WaitError;
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let waiter_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.current_notification_authority_is_live(
waiter_task,
notification_id,
expected_generation,
crate::ui::capability::CapabilityRights::NOTIFICATION_WAIT,
) {
return Err(crate::ipc::IpcError::InvalidCapability);
}
let waiter = NotificationWaiter::try_new(waiter_task, expected_generation, mask)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let mut notifications = crate::ui::capability::NOTIFICATION_REGISTRY.lock();
let target_index = notifications
.iter()
.position(|object| object.id() == notification_id)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let immediate = notifications[target_index].pending() & mask;
if immediate != 0 {
// S146 acceptance marker: IMMEDIATE_NOTIFICATION_DEADLINE=UNARMED.
let outcome = notifications[target_index]
.wait(waiter)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
let NotificationWaitOutcome::Immediate { observed } = outcome else {
panic!("ready notification bits published a waiter")
};
drop(notifications);
drop(deadlines);
drop(transaction);
return Ok(Some(observed));
}
if self.ready_queue.is_empty() {
return Err(crate::ipc::IpcError::NoReceiver);
}
self.save_current_ipc_context(ctx, user_sp);
let mut blocked = self.ipc_blocked_tasks.lock();
blocked
.try_reserve(1)
.map_err(|_| crate::ipc::IpcError::NoReceiver)?;
// NOTIFICATION_WAIT_ARM_BEFORE_WAITER: timer authority exists before
// the object can expose a parked waiter to a signal producer.
let deadline_record = deadlines
.register_notification(
waiter_task,
notification_id,
expected_generation,
mask,
now_tick,
timeout_ticks,
)
.map_err(|error| match error {
IpcCallDeadlineRegistryError::Wait(WaitError::TableFull) => {
crate::ipc::IpcError::QueueFull
}
IpcCallDeadlineRegistryError::Wait(
WaitError::DeadlineNotFuture | WaitError::DeadlineTooFar,
)
| IpcCallDeadlineRegistryError::WaitEpochExhausted => {
crate::ipc::IpcError::InvalidDeadline
}
IpcCallDeadlineRegistryError::Wait(_) => crate::ipc::IpcError::InvalidCapability,
})?;
match notifications[target_index].wait(waiter) {
Ok(NotificationWaitOutcome::Registered) => {}
Ok(NotificationWaitOutcome::Immediate { .. }) | Err(_) => {
deadlines
.cancel_exact(deadline_record)
.expect("failed notification waiter publication lost its deadline rollback");
return Err(crate::ipc::IpcError::InvalidCapability);
}
}
let mut task = self
.current_task
.take()
.expect("preflighted notification waiter task disappeared");
task.state = TaskState::BlockedOnNotification { notification_id };
let parked_context = &mut task.context as *mut TaskContext;
blocked.push(task);
drop(blocked);
drop(notifications);
drop(deadlines);
drop(transaction);
#[cfg(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5"))]
self.switch_after_ipc_park_with_membership_handoff(
parked_context,
|| {
drop(s377_writer_access);
drop(s377_irq_guard);
},
|scheduler| {
let s377_resume_irq_guard = crate::arch::aarch64::IrqGuard::new();
let s377_resume_writer_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s377_el0_notification_wait_timeout_writer_guard_integration::acquire_s377_production_scheduler_writer_access()
.unwrap_or_else(|error| {
panic!(
"S377 resumed EL0 notification-wait scheduler writer guard failed closed: {:?}",
error
)
});
scheduler.restore_current_ipc_context(ctx);
drop(s377_resume_writer_access);
drop(s377_resume_irq_guard);
},
);
#[cfg(not(all(target_arch = "aarch64", target_os = "none", feature = "board-rpi5")))]
{
self.switch_after_ipc_park(parked_context);
self.restore_current_ipc_context(ctx);
}
Ok(None)
}
/// OR one signal into a coalescing notification. A matching waiter is
/// validated across object, shared deadline, CNode and scheduler state
/// before the allocation-free signal/wake commit.
pub fn notification_signal(
&mut self,
notification_id: crate::ui::capability::CapId,
expected_generation: u64,
bits: u64,
) -> Result<crate::ipc_notification::NotificationSignalOutcome, crate::ipc::IpcError> {
use crate::ipc_notification::NotificationSignalOutcome;
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let _transaction = IPC_TRANSACTION_LOCK.lock();
let signaler_task = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if bits == 0
|| !self.current_notification_authority_is_live(
signaler_task,
notification_id,
expected_generation,
crate::ui::capability::CapabilityRights::NOTIFICATION_SIGNAL,
)
{
return Err(crate::ipc::IpcError::InvalidCapability);
}
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let mut notifications = crate::ui::capability::NOTIFICATION_REGISTRY.lock();
let target_index = notifications
.iter()
.position(|object| object.id() == notification_id)
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let waiter = notifications[target_index].waiter_snapshot();
let will_wake = waiter.is_some_and(|waiter| {
(notifications[target_index].pending() | bits) & waiter.mask() != 0
});
if !will_wake {
return notifications[target_index]
.signal(bits)
.map_err(|_| crate::ipc::IpcError::InvalidCapability);
}
let waiter = waiter.expect("will_wake without a notification waiter");
let deadline_record = deadlines
.task_snapshot(waiter.task_id())
.filter(|record| {
record.kind().tag() == crate::ipc_wait::WaitKindTag::Notification
&& record.kind().object_id() == notification_id
&& record.kind().object_generation() == waiter.capability_generation()
&& record.kind().notification_mask() == Some(waiter.mask())
})
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
let mut blocked = self.ipc_blocked_tasks.lock();
let task_position = blocked
.iter()
.position(|task| {
task.id == waiter.task_id()
&& matches!(
task.state,
TaskState::BlockedOnNotification {
notification_id: blocked_id,
} if blocked_id == notification_id
)
&& task
.cnode
.lookup_capability_by_id(notification_id)
.is_some_and(|capability| {
capability.kind == crate::ui::capability::CapabilityKind::Notification
&& capability.generation == waiter.capability_generation()
&& capability.rights.contains(
crate::ui::capability::CapabilityRights::NOTIFICATION_WAIT,
)
})
})
.ok_or(crate::ipc::IpcError::InvalidCapability)?;
if !self.ipc_wake_capacity_available(1) {
return Err(crate::ipc::IpcError::NoReceiver);
}
let outcome = notifications[target_index]
.signal(bits)
.map_err(|_| crate::ipc::IpcError::InvalidCapability)?;
let NotificationSignalOutcome::Wake {
waiter: committed_waiter,
observed,
..
} = outcome
else {
panic!("preflighted matching notification did not wake")
};
assert_eq!(committed_waiter, waiter);
let mut task = blocked.remove(task_position);
task.saved_user_gprs[0] = crate::ipc::IpcError::Ok.as_u64();
task.saved_user_gprs[1] = observed;
for register in &mut task.saved_user_gprs[2..=7] {
*register = 0;
}
task.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(task));
deadlines
.complete_delivery_exact(deadline_record)
.expect("matching notification signal lost exact deadline retirement");
Ok(outcome)
}
/// Commits a one-shot reply only after proving the caller is already in
/// the global blocked set. The reply object is removed after the response
/// is copied and the caller becomes runnable.
pub fn ipc_reply_commit(
&mut self,
reply_cap_id: crate::ui::capability::CapId,
message: crate::ui::capability::IpcMessage,
) -> crate::ipc::IpcError {
use crate::ipc_rendezvous::ReplyOutcome;
let _irq_guard = crate::arch::aarch64::IrqGuard::new();
let transaction = IPC_TRANSACTION_LOCK.lock();
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let Some(responder_task) = self
.current_task
.as_ref()
.map(|task| task.id)
.filter(|task_id| *task_id != 0)
else {
return crate::ipc::IpcError::InvalidCapability;
};
// Match lifecycle/mint lock order: IPC transaction -> capability
// provenance -> endpoint registry -> blocked scheduler set.
let mut capability_store = crate::ui::capability::get_capability_store();
let mut endpoints = crate::ui::capability::ENDPOINT_REGISTRY.lock();
let Some(reply_index) = endpoints
.iter()
.position(|endpoint| endpoint.id == reply_cap_id && endpoint.is_reply_cap)
else {
return crate::ipc::IpcError::InvalidCapability;
};
let caller_task = endpoints[reply_index].owner;
let Some(target_endpoint) = endpoints[reply_index].reply_target else {
return crate::ipc::IpcError::InvalidCapability;
};
let Some(target_index) = endpoints
.iter()
.position(|endpoint| endpoint.id == target_endpoint && !endpoint.is_reply_cap)
else {
return crate::ipc::IpcError::InvalidCapability;
};
let mut blocked = self.ipc_blocked_tasks.lock();
let Some(caller_position) = blocked.iter().position(|task| {
task.id == caller_task
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: true,
} if endpoint_id == reply_cap_id
)
}) else {
return crate::ipc::IpcError::InvalidCapability;
};
let reply_authority = blocked[caller_position]
.cnode
.lookup_capability_by_id(reply_cap_id)
.copied()
.filter(|capability| {
capability.owner == caller_task
&& capability.kind == crate::ui::capability::CapabilityKind::Endpoint
&& capability.parent.is_none()
});
let Some(reply_authority) = reply_authority else {
return crate::ipc::IpcError::InvalidCapability;
};
let deadline_record = deadlines.reply_snapshot(reply_cap_id);
if deadline_record.is_some_and(|record| {
record.key().task_id() != caller_task
|| record.kind().tag() != crate::ipc_wait::WaitKindTag::Call
|| record.kind().object_id() != target_endpoint
|| record.kind().reply_cap_id() != Some(reply_cap_id)
|| record.kind().reply_generation() != Some(reply_authority.generation)
}) {
return crate::ipc::IpcError::InvalidCapability;
}
if self.ready_queue.try_reserve(1).is_err() {
return crate::ipc::IpcError::NoReceiver;
}
match endpoints[target_index]
.rendezvous
.reply(responder_task, reply_cap_id, message)
{
Ok(ReplyOutcome::Wake {
caller_task: model_caller,
payload,
..
}) => {
if model_caller != caller_task {
panic!("reply model caller differs from blocked caller");
}
let retire_witness = endpoints[target_index]
.rendezvous
.retire(caller_task, reply_cap_id)
.expect("consumed reply record must retire exactly once");
assert_eq!(retire_witness.caller_task(), caller_task);
assert_eq!(retire_witness.reply_token(), reply_cap_id);
let mut caller = blocked.remove(caller_position);
assert_eq!(
caller.cnode.revoke_capability_exact(&reply_authority),
Some(reply_authority),
"retired reply exact caller CNode revoke failed"
);
Self::write_ipc_delivery(&mut caller, payload, 0);
caller.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(caller));
let removed_reply = endpoints.remove(reply_index);
assert_eq!(removed_reply.id, retire_witness.reply_token());
assert_eq!(removed_reply.owner, retire_witness.caller_task());
assert_eq!(removed_reply.reply_target, Some(target_endpoint));
if let Some(record) = deadline_record {
deadlines
.complete_reply_exact(record)
.expect("ordinary REPLY committed but exact deadline did not retire");
}
}
Ok(ReplyOutcome::StoredBeforePark) => {
// The blocked-set proof above and the model's `parked` bit
// must agree under the transaction lock. A recoverable error
// would leave a Replied record with a permanently parked
// caller, so treat disagreement as an integrity failure.
panic!("blocked caller has an unparked reply record")
}
Err(_) => return crate::ipc::IpcError::InvalidCapability,
}
drop(blocked);
drop(endpoints);
assert_eq!(
capability_store.revoke_endpoint_provenance(reply_cap_id, Some(responder_task)),
1,
"retired reply must have exactly one provenance record"
);
drop(capability_store);
drop(deadlines);
drop(transaction);
crate::ipc::IpcError::Ok
}
/// M7.5 — Revoke edildiğinde bloke task'leri uyandır
///
/// NOTE (M6/M7 Multi-core hazırlığı):
/// ipc_blocked_tasks şu anda Scheduler'ın içinde (tek global).
/// Gerçek SMP'de ya:
/// - Scheduler'ın tamamını spin::Mutex ile sarmak, veya
/// - Per-CPU blocked list + cross-CPU IPI ile wake.
///
/// Şu an için ENDPOINT_REGISTRY'nin yaptığı gibi .lock() ile korunması önerilir.
pub(crate) fn try_reserve_ipc_wake_capacity(&mut self, additional: usize) -> bool {
self.ready_queue.try_reserve(additional).is_ok()
}
/// Timer IRQ preflight must never allocate. All scheduler-created tasks
/// have already occupied the ready heap once, so a parked task can return
/// only when an existing slot is available. A false result leaves every
/// IPC/deadline object untouched for fail-closed handling.
pub(crate) fn ipc_wake_capacity_available(&self, additional: usize) -> bool {
self.ready_queue
.capacity()
.saturating_sub(self.ready_queue.len())
>= additional
}
/// Commit the scheduler half of one already-preflighted timeout. The
/// caller is selected by task id + reply object and its exact CNode
/// generation is retired before it becomes runnable with TimedOut.
pub(crate) fn wake_timed_out_ipc_caller_exact(
&mut self,
caller_task: u64,
reply_authority: crate::ui::capability::Capability,
) {
assert!(
self.ipc_wake_capacity_available(1),
"timed CALL wake lost preflighted ready capacity"
);
let mut blocked = self.ipc_blocked_tasks.lock();
let caller_position = blocked
.iter()
.position(|task| {
task.id == caller_task
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: true,
} if endpoint_id == reply_authority.id
)
&& task.cnode.can_revoke_capability_exact(&reply_authority)
})
.expect("preflighted timed CALL caller disappeared");
let mut caller = blocked.remove(caller_position);
assert_eq!(
caller.cnode.revoke_capability_exact(&reply_authority),
Some(reply_authority),
"timed CALL exact reply CNode revoke failed"
);
caller.saved_user_gprs[0] = crate::ipc::IpcError::TimedOut.as_u64();
for register in &mut caller.saved_user_gprs[1..=7] {
*register = 0;
}
caller.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(caller));
}
/// Wake one exact timed RECV without consuming its endpoint capability.
/// The timer-side graph preflight already removed the rendezvous waiter;
/// this commits only the allocation-free scheduler half.
pub(crate) fn wake_timed_out_ipc_receiver_exact(
&mut self,
receiver_task: u64,
endpoint_authority: crate::ui::capability::Capability,
) {
assert!(
self.ipc_wake_capacity_available(1),
"timed RECV wake lost preflighted ready capacity"
);
let mut blocked = self.ipc_blocked_tasks.lock();
let receiver_position = blocked
.iter()
.position(|task| {
task.id == receiver_task
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id,
is_call: false,
} if endpoint_id == endpoint_authority.id
)
&& task
.cnode
.lookup_capability_by_id(endpoint_authority.id)
.copied()
== Some(endpoint_authority)
})
.expect("preflighted timed RECV receiver disappeared");
let mut receiver = blocked.remove(receiver_position);
receiver.saved_user_gprs[0] = crate::ipc::IpcError::TimedOut.as_u64();
for register in &mut receiver.saved_user_gprs[1..=7] {
*register = 0;
}
receiver.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(receiver));
}
pub(crate) fn notification_blocked_task_count_on(
&self,
task_id: u64,
notification_id: crate::ui::capability::CapId,
) -> usize {
self.ipc_blocked_tasks
.lock()
.iter()
.filter(|task| {
task.id == task_id
&& matches!(
task.state,
TaskState::BlockedOnNotification {
notification_id: blocked_id,
} if blocked_id == notification_id
)
})
.count()
}
pub(crate) fn notification_blocked_count_on(
&self,
notification_id: crate::ui::capability::CapId,
) -> usize {
self.ipc_blocked_tasks
.lock()
.iter()
.filter(|task| {
matches!(
task.state,
TaskState::BlockedOnNotification { notification_id: blocked_id }
if blocked_id == notification_id
)
})
.count()
}
/// Wake one waiter whose notification authority was revoked after wait
/// admission. Holder removal may already have committed, so identity is
/// proven by task/object/state under the same IPC transaction rather than
/// by re-reading the deleted CNode slot.
pub(crate) fn wake_revoked_notification_exact(
&mut self,
waiter_task: u64,
notification_id: crate::ui::capability::CapId,
) {
assert!(
self.ipc_wake_capacity_available(1),
"revoked notification wake lost preflighted ready capacity"
);
let mut blocked = self.ipc_blocked_tasks.lock();
let task_position = blocked
.iter()
.position(|task| {
task.id == waiter_task
&& matches!(
task.state,
TaskState::BlockedOnNotification { notification_id: blocked_id }
if blocked_id == notification_id
)
})
.expect("preflighted revoked notification waiter disappeared");
let mut task = blocked.remove(task_position);
task.saved_user_gprs[0] = crate::ipc::IpcError::InvalidCapability.as_u64();
for register in &mut task.saved_user_gprs[1..=7] {
*register = 0;
}
task.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(task));
}
pub(crate) fn wake_timed_out_notification_exact(
&mut self,
waiter_task: u64,
notification_authority: crate::ui::capability::Capability,
) {
assert!(
self.ipc_wake_capacity_available(1),
"timed notification wake lost preflighted ready capacity"
);
let mut blocked = self.ipc_blocked_tasks.lock();
let task_position = blocked
.iter()
.position(|task| {
task.id == waiter_task
&& matches!(
task.state,
TaskState::BlockedOnNotification { notification_id }
if notification_id == notification_authority.id
)
&& task
.cnode
.lookup_capability_by_id(notification_authority.id)
.copied()
== Some(notification_authority)
})
.expect("preflighted timed notification waiter disappeared");
let mut task = blocked.remove(task_position);
task.saved_user_gprs[0] = crate::ipc::IpcError::TimedOut.as_u64();
for register in &mut task.saved_user_gprs[1..=7] {
*register = 0;
}
task.state = TaskState::Ready;
self.ready_queue.push(PriorityTask::new(task));
}
/// Allocation-free lifecycle audit: a parked rendezvous identity must map
/// to exactly one scheduler waiter before cancellation is committed.
pub(crate) fn ipc_blocked_count_on(&self, endpoint_id: crate::ui::capability::CapId) -> usize {
self.ipc_blocked_tasks
.lock()
.iter()
.filter(|task| {
matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id: blocked_endpoint,
..
} if blocked_endpoint == endpoint_id
)
})
.count()
}
pub(crate) fn ipc_blocked_task_count_on(
&self,
task_id: u64,
endpoint_id: crate::ui::capability::CapId,
is_call: bool,
) -> usize {
self.ipc_blocked_tasks
.lock()
.iter()
.filter(|task| {
task.id == task_id
&& matches!(
task.state,
TaskState::BlockedOnIpc {
endpoint_id: blocked_endpoint,
is_call: blocked_is_call,
} if blocked_endpoint == endpoint_id && blocked_is_call == is_call
)
})
.count()
}
pub fn wake_tasks_on_revoked_endpoint(&mut self, endpoint_id: crate::ui::capability::CapId) {
// Multi-core safe: PerCpu üzerinden kilitle
let mut deadlines = crate::ipc::IPC_CALL_DEADLINES.lock();
let mut blocked_list = self.ipc_blocked_tasks.lock();
let mut i = 0;
while i < blocked_list.len() {
let should_wake = matches!(
blocked_list[i].state,
TaskState::BlockedOnIpc { endpoint_id: eid, .. } if eid == endpoint_id
);
if should_wake {
let mut task = blocked_list.remove(i);
let task_asid = task.asid; // ASID'i kaydet (invalidate için)
let is_call = matches!(task.state, TaskState::BlockedOnIpc { is_call: true, .. });
if let Some(record) = deadlines.task_snapshot(task.id) {
match (is_call, record.kind().tag()) {
(true, crate::ipc_wait::WaitKindTag::Call) => assert_eq!(
record.kind().reply_cap_id(),
Some(endpoint_id),
"peer-close wake selected a different deadline reply"
),
(false, crate::ipc_wait::WaitKindTag::Receive) => assert_eq!(
record.kind().object_id(),
endpoint_id,
"peer-close wake selected a different deadline endpoint"
),
_ => panic!("peer-close wake selected a different IPC wait kind"),
}
deadlines
.complete_peer_closed_exact(record)
.expect("peer-close IPC wake lost its exact deadline record");
}
drop(blocked_list);
// M7 audit fix #10 + M8.2 ASID invalidate
task.saved_user_gprs[0] = crate::ipc::IpcError::InvalidCapability.as_u64();
for slot in &mut task.saved_user_gprs[1..=7] {
*slot = 0;
}
task.state = TaskState::Ready;
// Revoke sonrası ilgili ASID'in TLB girdilerini temizle (M8.2)
if task_asid != 0 {
unsafe {
crate::arch::aarch64::mmu::invalidate_asid(task_asid);
}
}
let task_id = task.id;
self.ready_queue.push(PriorityTask::new(task));
crate::kprintln!(
"[M7.5] Revoke nedeniyle task#{} uyandırıldı (ASID={} invalidate edildi)",
task_id,
task_asid
);
blocked_list = self.ipc_blocked_tasks.lock();
} else {
i += 1;
}
}
}
// Restored minimal versions of previously removed methods to make build pass
fn recalculate_min_vruntime(&mut self) {
// TODO: implement properly later
}
fn normalize_vruntime(&mut self) {
// TODO: implement properly later
}
pub fn current_vruntime() -> u64 {
#[cfg(feature = "board-rpi5")]
let _s255_scheduler_read_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s254_vruntime_read_access_guard_expansion::acquire_s255_production_scheduler_read_access()
.unwrap_or_else(|error| {
panic!(
"S255 current-vruntime scheduler read access failed closed: {:?}",
error
)
});
unsafe {
let sched = &*core::ptr::addr_of!(SCHEDULER);
sched.current_task.as_ref().map_or(0, |t| t.vruntime)
}
}
pub fn min_vruntime() -> u64 {
#[cfg(feature = "board-rpi5")]
let _s255_scheduler_read_access = crate::g8l_target_dispatch_scheduler_owner_scheduler_mutation_production_migration_lifecycle_s254_vruntime_read_access_guard_expansion::acquire_s255_production_scheduler_read_access()
.unwrap_or_else(|error| {
panic!(
"S255 min-vruntime scheduler read access failed closed: {:?}",
error
)
});
unsafe {
let sched = &*core::ptr::addr_of!(SCHEDULER);
sched.min_vruntime
}
}
pub fn try_wake_tasks_waiting_on(&mut self, _endpoint_id: crate::ui::capability::CapId) {
// TODO
}
}snippet sha256: fc5979057655…file sha256: 838dd474448c…
02 · Doğrulayan test kodu
Operations komutuna bağlı focused test
tam Rust öğesiL160–L168
simulation/tests/ipc_notification_lifecycle_runtime.rs::s147_preserves_abi_v1_3_and_does_not_add_a_syscall
#[test]
fn s147_preserves_abi_v1_3_and_does_not_add_a_syscall() {
assert!(ABI.contains("pub const ABI_MAJOR: u16 = 1"));
assert!(ABI.contains("pub const ABI_MINOR: u16 = 3"));
assert!(ABI.contains("pub const NOTIFICATION_SIGNAL: u64 = 17"));
assert!(ABI.contains("pub const NOTIFICATION_WAIT_TIMEOUT: u64 = 18"));
assert!(!ABI.contains("S147"));
}snippet sha256: d9365d324e21…file sha256: 279d415f7118…
03 · Kapı kimlik kaydı
Operations sıra, kimlik ve başlık bağı
tam Operations kaydıL24389–L24490
website/src/lib/operations.ts::k2-notification-lifecycle-revoke-partial
{
id: "k2-notification-lifecycle-revoke-partial",
date: "2026-08-24",
sequence: 147,
status: "passed",
umbrella_status: "partial",
title: "K2: Notification lifecycle ve post-admission revoke",
summary:
"S147, ABI v1.3'ü ve S146 syscall numaralarını değiştirmeden iki production lifecycle kenarını kapatır. Aktif notification WAIT grant revoke edildiğinde exact waiter/deadline authority Cancelled olarak emekliye ayrılır; notification owner task lifecycle teardown'ında object, derived grant, waiter ve deadline PeerClosed olarak kaldırılır. İki strict RuntimePmm EL0 waiter da exact bir kez InvalidCapability ile uyanır; stale WAIT ve stale SIGNAL reddedilir. Concurrent üç-yollu signal/revoke/timeout fault injection ve Generic SMP açık olduğundan K2 PARTIAL kalır.",
evidence: [
"Fail-closed kaynak kapısı: QEMU lifecycle fixture/smoke eksikken 9/10 RED → final S147 notification lifecycle source/policy 10/10 PASS.",
"ABI UNCHANGED_V1_3: yeni syscall yok; SYS_NOTIFICATION_SIGNAL=17 ve SYS_NOTIFICATION_WAIT_TIMEOUT=18 aynen korunur.",
"S129–S147 exact envanteri 36 binary / 216 testtir; 216/216 PASS. Ortak ABI/IPC odaklı kapı 8 binary / 78/78 PASS.",
"Grant revoke exact preflight/commit'i aktif deadline'ı Cancelled yaptı; owner task lifecycle exact object/grant/wait teardown'ı deadline'ı PeerClosed yaptı.",
"QEMU notification=52/53, waiter=57/58, domain=1330597193/1330597194; REGISTERED=2, CANCELLED=1, PEER_CLOSED=1, TIMED_OUT=0, DELIVERED=0, REPLIED=0.",
"REVOKED_WAKE=INVALID_CAPABILITY ve OWNER_CLOSE_WAKE=INVALID_CAPABILITY; STALE_WAIT=REJECTED, STALE_SIGNAL=REJECTED, EXACT_ONCE=YES ve BOTH_STRICT=YES.",
"OWNED_NOTIFICATION_CLEANUP=2, DERIVED_GRANT_CLEANUP=1, reclaim=10, free 6139→6129→6139, active 5→15→5, KERNEL_FAULTS=0 ve EXECUTOR=PASS.",
"QEMU strict ELF W^X 28/28, başarılı IPC reply 20/20 ve mevcut RuntimePmm/scheduler regresyonları PASS.",
"Manuel 8 saniyelik QEMU log'u 473667 B / SHA-256 8bca87711e53354b4b29f6b1cd92b703a33c69c4035af59c997dee4446d136e1 olarak ölçüldü.",
"AArch64 board-qemu, board-rpi4, board-rpi5 ve board-rpi5+smp applicability 4/4 PASS.",
"Tam workspace yedi S147-dışı frozen G8h identity/closure assertion'ında kırmızıdır. Exact yedisi dışlanınca kalan 108 result group / 688 test 688/688 PASS; assertion'lar gevşetilmedi ve full GREEN iddia edilmez.",
"Fiziksel operatör sırası görünürdür: Gücü kapat → SD kartı Pi'den çıkar → SD kartı Mac'e tak → yetkili write/verify/read-back işlemini tamamla → SD kartı Mac'ten güvenli çıkar → SD kartı güçsüz Pi'ye tak → UART capture pre-arm ve exact identity kapısını doğrula → Güç ver.",
"Bu fiziksel sıra S147'de uygulanmadı: physical/device operations=0, RUNBOOK_EXECUTED=NO, S124 archive/promotion STOP, son fiziksel boot/runtime PASS S92 BOOT8G ve son storage/media PASS S119.",
"Güncel planlama tahmini S147 bazında R2 bitiş aralığı S331–387, risk-paylı merkez ≈S364'tür; taahhüt veya fiziksel PASS değildir ve yeni bir S148 işlemi oluşturmaz.",
"Yerel web kapıları: içerik 243/243, ESLint, TypeScript --noEmit ve Next.js static export 23/23 route / 191 file PASS; operations/timeline/yol-haritasi üzerinde S147 ve exact fiziksel operatör marker matrisi 27/27'dir.",
"Kalıcı kapsam: `docs/K2-S147-Notification-Lifecycle-Revoke-Proof.md`.",
],
commands: [
"cargo test -p aselsan_microkernel_simulation --test ipc_notification_lifecycle_runtime -- --test-threads=1",
"cargo test -p aselsan_microkernel_simulation [36 exact focused test binary] -- --test-threads=1",
"cargo check -p aselsan_kernel --target aarch64-unknown-none [board-qemu, board-rpi4, board-rpi5, board-rpi5+smp]",
"make verify-qemu",
"cargo test --workspace -- --test-threads=1",
"cargo test --workspace -- --test-threads=1 [seven exact historical --skip filters]",
"python3 scripts/render-project-status.py --check",
"cd website && npm test && npm run lint && npx tsc --noEmit && npm run build",
],
terminalSessions: [
{
id: "s147-red-before-lifecycle-runtime",
title: "Notification lifecycle kaynak kapısı: fail-closed bring-up",
commandLines: [
"cargo test -p aselsan_microkernel_simulation --test ipc_notification_lifecycle_runtime -- --test-threads=1",
],
outputLines: [
"QEMU lifecycle fixture/smoke missing: 9/10 RED",
"no partial PASS accepted",
"final source/policy gate after runtime integration: 10/10 PASS",
],
exitCode: 101,
outputMode: "selected",
},
{
id: "s147-green-focused-aarch64-qemu",
title:
"Exact grant revoke, owner lifecycle, stale authority reddi ve QEMU",
commandLines: [
"cargo test -p aselsan_microkernel_simulation [36 exact focused test binary] -- --test-threads=1",
"cargo check -p aselsan_kernel --target aarch64-unknown-none [4 profiles]",
"make verify-qemu",
],
outputLines: [
"ipc_notification_lifecycle_runtime: 10/10 PASS",
"combined exact focused inventory: 216/216 PASS · 36 binaries",
"focused ABI/IPC: 78/78 PASS · AArch64 compile profiles: 4/4 PASS",
"[K2-S147] STRICT EL0 NOTIFY WAKE=INVALID_CAPABILITY · exact count 2",
"[K2-S147] STRICT EL0 NOTIFY STALE_WAIT=REJECTED · exact count 2",
"[K2-S147] ABI=UNCHANGED_V1_3 GRANT_REVOKE=EXACT OWNER_LIFECYCLE=EXACT REGISTERED=2 CANCELLED=1 PEER_CLOSED=1 STALE_SIGNAL=REJECTED EXACT_ONCE=YES BOTH_STRICT=YES RECLAIM=10 OWNED_NOTIFICATION_CLEANUP=2 DERIVED_GRANT_CLEANUP=1 free=6139->6129->6139 active=5->15->5 KERNEL_FAULTS=0 EXECUTOR=PASS",
"QEMU smoke PASS: strict ELF W^X 28/28 · IPC reply 20/20 · S147 PASS",
],
exitCode: 0,
outputMode: "selected",
},
{
id: "s147-workspace-independent-history-red",
title:
"Tam workspace: yedi S147-dışı frozen G8h identity/closure kırmızısı",
commandLines: [
"cargo test --workspace -- --test-threads=1",
"cargo test --workspace -- --test-threads=1 [seven exact historical --skip filters]",
],
outputLines: [
"full workspace: FAIL-CLOSED · 7 historical frozen G8h identity/closure assertions",
"raw run stops at the independent S96 exceptions.S SHA assertion",
"excluding exactly those seven assertion names: 108 result groups · 688/688 PASS",
"historical assertions were not relaxed · full-workspace GREEN is not claimed",
],
exitCode: 101,
outputMode: "selected",
},
],
terminalSessionsNote:
"S147 exact notification grant revoke ve owner/task lifecycle teardown dilimlerini kapatır. Concurrent üç-yollu signal/revoke/timeout fault injection ve Generic SMP açık olduğundan K2 COMPLETE değildir. Fiziksel runbook açıkça görünür, fakat uygulanmadı.",
limitations: [
"Concurrent signal/revoke/timeout üç-yollu fault-injection matrisi kapanmadı.",
"Allocation-free wait/deadline registry ürün kapasite ve latency bütçeleri imzalı değildir.",
"Generic SMP current-task stop/migration, cross-CPU timer/signal/wake/TLB/reaper, capability transferi, shared-memory loan ve ortak reconciliation açıktır.",
"Cross-subsystem rollback, diğer fault/concurrency sınıfları ve signed product thresholds açıktır.",
"Full workspace yedi S147-dışı frozen G8h identity/closure assertion'ı nedeniyle GREEN değildir; exact yedi assertion dışlandığında kalan 688/688 PASS'tir.",
"S124 fiziksel archive/promotion STOP; görüntülenen güç/SD/Mac/UART sırası S147'de yürütülmedi.",
],
},snippet sha256: e34a322f5787…file sha256: 9726dbf00f84…
Focused test komutu
cargo test -p aselsan_microkernel_simulation --test ipc_notification_lifecycle_runtime -- --test-threads=1proof: docs/K2-S147-Notification-Lifecycle-Revoke-Proof.md
Registry schema v5 · generator
website/scripts/generate-code-gates.mjs · Tam SHA-256: 3050638b71a684d8f8f947a8a6faa237a17fa8db5dc0db04fb207b668b462af9