Methodology

Planned — not yet executed. The pilot described below is scheduled; results will be published after the intervention is complete.

Approach and scope

E-DIGNA adopts a mixed-methods explanatory sequential design (Creswell & Creswell, 2018): quantitative data first — surveys and a performance test, before and after — then qualitative data whose job is to explain the numbers.

The scope is pre-experimental, with a single group measured before and after.

There is no comparison group, and that absence is declared as a feature of the design rather than an oversight: withholding fraud-protection training from one class in order to use it as a control was not defensible. The methodological consequence is accepted head-on — the study can document change, not net effect, and makes no causal claim.

The competence framework is DigComp 2.2 (Vuorikari, Kluzer & Punie, 2022), complemented by UNESCO’s 2024 AI competency frameworks for students and teachers.

Why a practice-first design

The decision follows the available evidence. The most recent meta-analysis of cybersecurity training for end users, covering 69 studies, finds a positive overall effect (d = 0.75) and a particularly strong one on predictors of behaviour — knowledge and attitudes — at d = 1.02. The decisive figure is the other one: in studies measuring observed behaviour, the effect drops to d = 0.36, with a confidence interval that includes zero (Prümmer, van Steen & van den Berg, 2024).

Put plainly: it is easy to get someone to know what phishing is, and hard to get them not to click.

That finding shaped the whole design. Since declared knowledge improves easily, measuring only that would be handing ourselves a good result; hence the inclusion of a situational performance test, as argued by work on awareness-programme metrics (Chaudhary, Gkioulos & Katsikas, 2022).

Instruments (planned)

1. Diagnostic survey (pre) — knowledge, habits and self-efficacy, with items anchored in DigComp 2.2.
2. Post survey — the same items, plus perceived usefulness and applicability.
3. Situational performance test — ten simulated messages the student classifies as legitimate or fraudulent, with justification. Measures behaviour, not self-perception.
4. Classroom observation rubric — on the use of the five tools. No personal data.
5. Closing focus group — semi-structured guide; the phase that explains the why behind the numbers.
6. Researcher field diary.
7. Outreach talk rubric for the awareness session with younger sections.

Pilot schedule (5 weeks)

Week	Activity	Purpose and instruments
1	Exploration and diagnosis	Baseline. Pre survey + performance test (form A)
2	Fundamentals and account protection	Password checker and digital mirror
3	Threats and simulation	Phishing simulator and EXIF viewer
4	Analysis, cases and debate	Technoethical reflection and digital resilience
5	Assessment and closing	Post survey + performance test (form B) + focus group

Analysis

Descriptive statistics; for the pre/post comparison, Wilcoxon signed-rank test (or paired t if normality holds), reporting effect size. Performance-test results are analysed separately from self-report, precisely because the evidence shows they move differently. Thematic coding for qualitative material.

Ethics

Participants are minors, and a project teaching digital dignity is judged first by how it treats their data.

• Informed consent from the guardian and assent from the student, revocable at any time with no academic consequence.
• Anonymity: instruments collect no name or ID. Pre/post matching uses a code the student generates, which no one can reverse.
• Aggregate reporting throughout; data handling under Colombia’s Law 1581 of 2012 and its reinforced regime for minors.
• No identifiable faces published without express authorisation.
• Fictional fraud scenarios: no real institutional case is re-enacted and no student is asked to recount their own victimisation. If a real case emerges, the institutional protocol under Law 1620 of 2013 is activated.
• Product coherence: the tools store and transmit nothing. The same requirement the project makes of its research, applied to its software.

Declared limitations

Without a comparison group no causality is claimed; the sample is a convenience sample at a single institution; part of the measurement is self-report and subject to social desirability; the researcher is also the workshop leader, a role conflict mitigated by anonymous instruments and teacher accompaniment but not eliminated; and five weeks show immediate change, not whether it lasts.

---

Full translation in progress — complete Spanish version available at /investigacion/metodologia.