Skip to content
BoKSA

CYBOK 04.6: Software Developers and Usable Security

CYBOK 04.6: Software Developers and Usable Security

Relevant learning outcome

CYBOK topic Learning outcome
04.6 Software developers and usable security Analysis

Within this BOKSA mapping, this article supports Analysis. A redesign or implementation can be a valuable next step, but you must connect that work to an additional BOKSA topic and provide evidence for the corresponding learning outcome.

Examples of possible products

  • Developer security task and workflow analysis
  • Security API or library usability assessment
  • Evaluation of security documentation and code examples
  • Analysis of security warnings from an IDE, scanner or CI/CD pipeline
  • Root-cause analysis of recurring security mistakes in a development environment
  • Comparative study of two libraries, APIs, tools or information sources
  • Research report on developer workload, security decisions and support needs

These products investigate why secure development succeeds or fails. A vulnerability scan or secure-coding checklist on its own is not sufficient evidence for this Human Factors topic because it does not analyse the interaction between developers, their tasks and their environment.

Why this matters

Developers are users of programming languages, frameworks, libraries, application programming interfaces, documentation and security tools. Technical expertise does not remove human limitations. Developers work under time pressure, switch between many systems and usually focus first on delivering functional software. Security is often a secondary task unless the requirement, tool or workflow makes it visible.

Poorly designed security APIs and tools can make insecure code easy to produce and secure code difficult to discover. A small misunderstanding can then be repeated across products and affect many end users. Analysing developer-facing usability helps you move beyond explanations such as "the developer should have known better" and identify conditions that can be changed.

In this article, security API includes the interfaces that developers use for authentication, authorisation, cryptography, password storage, secrets, logging and other security functions. It is not limited to the security of REST or web APIs.

What you analyse

The development task and its context

Begin with the developer's primary goal. This may be implementing registration, releasing a feature or resolving a production defect. Then identify where security enters that task. A security requirement that exists only in a policy document may not be visible at the moment a developer chooses a library or copies an example.

Relevant contextual factors include:

  • deadlines, interruptions and parallel work;
  • experience with the language, framework and security concept;
  • team conventions, code review and access to specialists;
  • the IDE, build pipeline and available automation;
  • official documentation, tutorials, forums and generated code;
  • performance targets and incentives;
  • the consequences of a warning or build failure.

API and library usability

Do not assess an API only by asking whether a secure implementation is technically possible. Analyse whether the secure path is discoverable, understandable and efficient for the intended developer. Useful questions include:

  • Is the secure default also the easiest starting point?
  • Does the developer have to make choices that require specialist knowledge?
  • Are dangerous or obsolete options easy to select accidentally?
  • Do names, parameters and types communicate their security meaning?
  • Are errors clear, specific and recoverable?
  • Do examples show complete, functional and secure use?
  • Can the developer test whether the implementation is correct?

Documentation and information sources

Developers often choose information that helps them complete a task quickly. Official documentation may be more secure but harder to apply, while an informal answer may be easy to copy but incomplete or outdated. Treat this as a productivity-security trade-off to investigate, not as a reason to forbid a resource without providing a workable alternative.

Check which sources developers actually use, how they evaluate trust, whether examples compile and whether security assumptions are explained. Also record version and date information: an answer that was secure for an older framework may now be unsafe.

Security tools and warnings

Static analysis, dependency scanners, IDE plug-ins and CI/CD gates are also user interfaces. High false-positive rates, vague messages and findings that arrive too late can cause warning fatigue. A useful finding should explain the problem in the developer's context, identify the affected code, support a secure next step and appear at a point where the developer can act.

Do not confuse the number of warnings with tool effectiveness. Analyse whether developers notice, understand, trust and correctly resolve the findings.

A practical starting point

1. Formulate a focused analysis question

Connect one developer task to a security outcome. For example:

How do Java developers select and use a password-hashing library when implementing account registration under normal project constraints?

This is stronger than "Are developers security aware?" because the task, technology and expected result can be observed.

2. Define secure and functional success

Establish trusted criteria before observing participants or artefacts. Functional correctness and security correctness are different. Code may run successfully while using an obsolete algorithm, disabling certificate validation or exposing a secret.

Use relevant standards, vendor documentation or reviewed security guidance to define the expected secure result. Record versions and assumptions so another person can reproduce your judgement.

3. Select representative participants and tasks

Match experience, role, language and framework to the environment you are studying. A small study with realistic participants is usually more useful than a large study with an unrelated group. If access to developers is limited, clearly state what your sample can and cannot represent.

4. Map the workflow and information environment

Identify the point at which the developer makes a security-relevant decision. Record the tools, documentation, examples, team interactions and time available. Contextual inquiry, observation, interviews and task walkthroughs can reveal different parts of the workflow.

5. Evaluate the interaction

Choose methods that fit your question. Options include:

  • a cognitive walkthrough of an API or tool;
  • a think-aloud programming task;
  • comparison of documentation and code examples;
  • analysis of IDE or CI/CD warning messages;
  • observation of code review and help-seeking;
  • analysis of recurring mistakes in repositories or support questions;
  • task-completion time, error rates and workload measures.

Avoid changing several variables at once unless your research design can separate their effects.

6. Triangulate the evidence

Combine what developers say with what they do and what the resulting artefact contains. An interview can explain a decision, but source code or task observation is needed to check the result. Tool logs can show timing and frequency but may not explain the reason.

7. Explain causes without blaming individuals

Separate lack of knowledge from poor discoverability, misleading examples, unsafe defaults, time pressure, weak feedback and organisational constraints. A finding such as "four participants selected the first working example" is evidence. Calling them careless is an unsupported judgement.

8. Report limitations and transferability

Explain the boundaries of the analysis: sample size, participant experience, artificial tasks, versions, time limits and access to normal support. Do not generalise from students to professional developers, or from one library to an entire ecosystem, without justification.

Short example: analysing password storage

A team asks developers to implement user registration. The requirement mentions account creation but does not explicitly mention password storage. Several implementations store passwords insecurely.

A Human Factors analysis does more than identify the defect. It examines when security became visible in the task, which examples and libraries developers found, what the defaults encouraged, whether feedback exposed the mistake and how deadline pressure affected the choice. You could compare a general task prompt with one that makes the security requirement explicit, or compare two documentation sets while checking both functional and security outcomes.

The result is an evidence-based explanation of the interaction between the developer, task and development environment. The implementation defect is important, but it is not the whole analysis.

Minimum evidence to collect

Question Possible evidence
What was the developer trying to achieve? Task description, observation, interview, issue or user story
Where did a security decision occur? Workflow map, screen or code recording, commit history
What information and support were available? Documentation, examples, search path, team practices
Was the result functionally correct? Tests, build result, task completion
Was the result secure? Trusted criteria, security tests, expert review
What made the task easier or harder? Errors, time, workload, think-aloud data, interview evidence
How transferable are the findings? Participant, task, tool and version descriptions; study limitations

Quality checklist

  • Is the analysis centred on a specific developer task and security outcome?
  • Did you distinguish security API usability from general web API security?
  • Are functional correctness and security correctness evaluated separately?
  • Are the secure criteria based on trustworthy and version-appropriate sources?
  • Are participants, tasks, tools and constraints representative of the stated context?
  • Did you examine defaults, terminology, parameters, errors, examples and recovery?
  • Did you include the information sources and support channels developers actually use?
  • Are warning false positives, timing, clarity and actionability considered?
  • Did you combine self-reported evidence with observed behaviour or artefact analysis?
  • Are knowledge gaps distinguished from usability and organisational barriers?
  • Does the report avoid blame and unsupported claims about developer motivation?
  • Are versions, research procedures, limitations and ethical considerations documented?

Further learning

For advanced students Level 3

Advanced analysis can move from a one-off usability review to empirical research on developer behaviour. You might compare APIs through a controlled programming study, analyse usability smells across a library ecosystem or study how security findings move through a real development pipeline over time.

Possible advanced products include:

  • a controlled study comparing the functional and security outcomes of two APIs;
  • a thematic analysis of developer questions and recurring usability smells;
  • a longitudinal analysis of how developers respond to scanner findings;
  • an ecological-validity review comparing a laboratory task with professional practice;
  • a reproducible benchmark for secure API or documentation usability.

Advanced work must treat study design as part of the security analysis. Control relevant variables, predefine success criteria where possible and explain uncertainty. Repository mining and developer telemetry can expose sensitive information, so minimise collected data, obtain appropriate permission and avoid using individual performance data for punitive purposes.