NUP for Probabilistic Software
Probabilistic software—including large language models, generative AI pipelines, diffusion networks, and autonomous agents—behaves fundamentally differently from deterministic code. In probabilistic computing, variability is an essential capability rather than a defect: identical inputs yield valid alternative outputs, model behavior evolves through context, and systems exhibit emergent conduct. NUP for Probabilistic Software provides rigorous engineering practices to build AI-native systems alongside the governance frameworks required to keep them trustworthy, auditable, and compliant with standards like NIST AI RMF 1.0, ISO/IEC 42001, and the EU AI Act.
8 Core Curriculum Disciplines
Comprehensive engineering tracks covering foundations, model interfaces, memory, retrieval, toolkits, and communications.
Core Concepts
Foundations of probabilistic computing, stochastic evaluation, failure modes, and lifecycle governance.
The Four Layers of LLM Engineering
Prompt, Context, Harness, and Loop Engineering layers for AI-native architectures.
AI Context Playbooks
Playbooks and shift-left guidance across all engineering roles.
Trust & Retrieval Engineering
Enterprise RAG pipelines, retrieval strategies, safety guardrails, and hallucination mitigation.
AI-Native Technical Communications
Core principles of writing for AI and embedding context.
AI-Native Tech Stack Engineering
Infrastructure and backend design for AI gateways and text-to-SQL middleware.
Content Engineering & SEO Automation
Automated SEO and validation pipelines.
Agentic Engineering Toolkits
Toolkits for SDD, OpenSpec, and Claude Code.
Copy this prompt into your AI coding assistant to assess your microservices for stochastic failure modes.
The Probabilistic-First Computing Paradigm
Deterministic software engineering operates on the axiom that a fixed input into a pure function yields an identical output every execution. Probabilistic systems violate this core assumption: the same prompt presented to a model can produce diverse outputs across runs. Rather than attempting to force artificial determinism upon probabilistic models, NUP treats stochastic variation as a first-class property governed by statistical envelopes, continuous calibration, and runtime guardrails.
Fundamental Uncertainty
Probabilistic systems operate across latent token spaces where generative outputs are sampled from probability distributions. Uncertainty is an inherent mathematical property, not a software bug or defect.
Non-Repeatability
Variability is a core feature enabling synthesis, creative problem-solving, and adaptability. Exact string equality assertions are replaced by semantic evaluation and semantic equivalence bands.
Distribution-Based Correctness
A single isolated unexpected output does not signify system failure, just as a single perfect response does not prove safety. High assurance requires evaluation across statistical sample populations with confidence intervals.
Emergent Behavior
Because LLMs and neural architectures learn multi-modal patterns rather than following hardcoded logic trees, engineering shifts from authoring control flows to context engineering, harness orchestration, and loop governance.
Why Traditional SDLC Fails for Probabilistic Systems
Traditional CI/CD pipelines assume binary pass/fail assertions. When testing probabilistic systems, a prompt that succeeds 98% of the time will fail conventional unit tests intermittently, resulting in test disabling or false confidence.
| Dimension | Deterministic SDLC | Probabilistic NUP |
|---|---|---|
| Unit Testing | Rigid assertions comparing exact output values against fixed fixtures. | Statistical assertions evaluating distributions, embedding similarity, and LLM-as-judge scoring. |
| Regression Testing | Replaying recorded inputs to verify identical byte-for-byte outputs. | Regression evaluation against curated benchmark datasets with drift thresholds. |
| Code Coverage | Measuring line, branch, and statement execution metrics (e.g. 90% branch coverage). | Evaluating prompt coverage, context permutation testing, and edge-case red teaming. |
| Point-in-Time Validation | One-time validation before release; software remains static until next deployment. | Continuous post-deployment monitoring for model drift, retrieval decay, and data changes. |
| Error Handling | Catching explicit exceptions, null pointers, and HTTP error status codes. | Detecting subtle hallucinations, out-of-domain queries, refusals, and prompt injection attacks. |
The 6-Stage NUP Lifecycle for Probabilistic Software
NUP organizes probabilistic development into six sequential, audit-ready phase gates. Each phase produces deterministic audit evidence to ensure safety and regulatory compliance.
NUP 6-Stage Probabilistic Lifecycle Explorer
Discovery & Problem Framing
Defines whether the problem requires probabilistic AI or traditional deterministic algorithms. Assesses data quality, regulatory exposure (FDA SaMD, EU AI Act risk tiers), and establishes statistical success metrics.
Primary Engineering Activities
- Evaluate feasibility: Probabilistic vs. Deterministic problem categorization
- Conduct initial AI risk classification (EU AI Act Prohibited / High-Risk / Minimal)
- Assess training/context data availability, copyright provenance, and representation bias
- Define non-deterministic success criteria (acceptable error rates, latency budgets, cost per token)
- Plan knowledge transformation and data extraction requirements from source documents
Probabilistic Failure Risks
- Applying LLMs to problems requiring 100% exact math or deterministic business rules
- Underestimating latency and token consumption at enterprise scale
- Failing to identify regulatory high-risk classification before architecture freeze
- Approved Problem Framing Brief with explicit probabilistic justification
- Initial AI Risk Classification Matrix signed off by compliance officer
- Target statistical metrics defined (Target Accuracy ≥ 95%, Hallucination Rate ≤ 1%)
Regulatory Framework Alignment
Probabilistic systems deployed in healthcare, finance, and critical infrastructure must satisfy rigorous multi-jurisdictional compliance mandates.
Traditional & AI-Specific Regulatory Alignment
| Framework & Standard | Governing Scope | Core Requirement | Auditable Artifact |
|---|---|---|---|
EU AI Act (Regulation 2024/1689)European Union | All AI systems placed on the EU market across 4 risk tiers | Risk management system, data governance, technical documentation, record-keeping, human oversight, cybersecurity. | Technical Documentation Dossier (Annex IV), EU Declaration of ConformityEU AI Act Articles 9, 10, 11, 14, 15 |
NIST AI RMF 1.0National Institute of Standards and Technology (USA) | Voluntary framework for managing risks in design, development, and use of AI | Four core functions: GOVERN, MAP, MEASURE, MANAGE across trustworthiness characteristics. | AI Risk Management Playbook & Trustworthiness ProfileNIST Special Publication 1270 / AI 100-1 |
ISO/IEC 42001:2023ISO / IEC | Certifiable AI Management System (AIMS) standard for organizations | Systemic organizational AI governance, risk assessment, continuous monitoring, and supplier management. | Statement of Applicability (SoA) & AI Management System ManualISO/IEC 42001:2023 Clauses 6, 8, 9, 10 |
FDA AI/ML SaMD & PCCP GuidanceUS Food and Drug Administration (FDA) | Software as a Medical Device incorporating machine learning algorithms | Good Machine Learning Practice (GMLP) and Predetermined Change Control Plans for planned model evolution. | Predetermined Change Control Plan (PCCP) & Clinical Performance Validation ReportFDA Docket FDA-2022-D-2628 / GMLP 10 Principles |
IEEE 7000-2021IEEE Computer Society | Model process for addressing ethical concerns during system design | Eliciting ethical values, translating them into system requirements, and risk-assessing ethical impacts. | Ethical Value Register & Impact Assessment DocumentIEEE Standard 7000-2021 Clause 5 |
HIPAA Security RuleUS Department of Health & Human Services | AI systems processing Electronic Protected Health Information (ePHI) | Access control, encryption, data de-identification, business associate agreements with LLM providers. | PHI Data Flow Diagram & LLM Zero-Data-Retention Compliance Attestation45 CFR Part 160 & Part 164 Subparts A/C |
NIST CSF 2.0NIST | Cybersecurity risk management applied to AI infrastructure and models | Protection against prompt injection, model theft, training data poisoning, and unauthorized inference access. | AI System Cybersecurity Assessment & Threat ModelNIST Cybersecurity Framework 2.0 (Govern, Identify, Protect, Detect, Respond, Recover) |
Required Continuous Operational Evidence Artifacts
| Evidence Artifact | Category & Cadence | Governance Purpose | Primary Quality Metric |
|---|---|---|---|
Operational Inference Metrics | OperationalContinuous (Real-time telemetry) | Tracks latency, token throughput, cache hit ratio, and per-query operational costs over time. | p95 Latency < 1.2s, Cache Hit Rate > 45% |
Output Distribution Curves | StatisticalDaily aggregate sweeps | Characterizes semantic distributions and detects distribution shifts away from validated baseline datasets. | KL Divergence / Wasserstein Distance < 0.05 |
Error Classification Logs | SafetyHourly automated classifier | Categorizes failures into factual hallucinations, safety refusals, out-of-domain errors, and schema mismatches. | Hallucination Rate < 0.8%, Schema Error = 0.0% |
User Feedback Aggregations | OperationalWeekly synthesis reports | Collects implicit and explicit thumbs up/down, user prompt rewrites, and escalation rates. | Positive Rating > 92%, Escalation Rate < 3% |
Safety Incident & Near-Miss Logs | SafetyInstantaneous incident logging | Documents prompt injections, jailbreak attempts, harmful output generations, and corrective patches applied. | Zero Uncontained Severity-1 Safety Events |
Semantic & Retrieval Drift Reports | GovernanceMonthly formal audit report | Compares monthly production sample outputs against frozen golden evaluation benchmarks. | Golden Benchmark Score Stability ± 1.5% |
Community Discussion & Feedback
Attributed peer feedback and official Netspective architecture notes.