Please note the ANZCTR will be unattended from 21/12/2018 until 08/01/2019. During this time no submissions or updates will be processed. Please ensure all trials are submitted/updated as soon as possible if you wish for them to be reviewed prior to the unattended period.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12617000017325
Ethics application status
Approved
Date submitted
16/12/2016
Date registered
5/01/2017
Date last updated
7/08/2018
Type of registration
Retrospectively registered

Titles & IDs
Public title
Evaluation of NetworkZ: Can multidisciplinary team-training improve the safety of surgery in New Zealand?
Scientific title
Evaluation of the NetworkZ programme: Can multidisciplinary surgical simulation team-training improve teamwork processes and the safety of surgery in New Zealand?
Secondary ID [1] 290450 0
Nil known
Universal Trial Number (UTN)
U1111-1189-3992
Trial acronym
NetworkZ
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Preventable surgical harm and death 300820 0
Condition category
Condition code
Surgery 300645 300645 0 0
Other surgery
Anaesthesiology 300646 300646 0 0
Other anaesthesiology

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
NetworkZ is team-training intervention designed to improve information sharing between team members, improve communication strategies and develop mutual respect and trust between Operating Room (OR) staff.
The educational framework used is the Salas Model of effective teams. Whole operating room teams participate in immersive, simulated surgical cases where the challenges to teamwork and communication are amplified in order to demand optimal processes and expose deficiencies. Each simulation is followed by a structured debrief, led by trained and accredited simulation instructors, exploring reactions, clarifying scenario details to ensure a shared understanding of what happened, exploring what actually happened and why, and drawing lessons on communication and teamwork to apply to clinical practice. The debrief lasts approximately 45 minutes. Simulation based training is supported by communication skills training comprising four separate 30 minute workshops on the principles of teamwork and shared mental model; speaking up; closed loop communication; and structured recap. Workshops include videos, slide presentations and small group discussion. The communication skills training is delivered by trained and accredited NetworkZ instructors at the DHB.
The simulator used is a 3G Laerdal Simman, set in a simulated or real operating room environment, and integrated with realistic custom-designed surgical models of trauma or pathology requiring a specific clinical response. All patient notes, laboratory investigations, equipment, drugs, fluids and consumables such as surgical drapes, gowns, gloves and masks will be provided as in a real theatre case.
The simulated cases will be modelled on real life cases, and include up to 25 different surgical scenarios in the disciplines of general surgery, ENT/ORL, urology, plastics and orthopaedics. A full team training intervention comprises 1 full day of simulations, debriefs and discussion. In each day, there will be 4-5 simulations and debriefs.
Simulations will be chosen from the 25 available scenarios based on the participants attending the session. As nursing, technician and anaesthesia staff work across all of the surgical disciplines portrayed in the scenarios. choice of scenario will be based on the discipline of the surgeon or surgeons attending any particular course, also ensuring that participants have not previously been exposed to that scenario.
The intervention will be delivered at each District Health Board (DHB) in New Zealand and may be run either in existing operating theatres or in a purpose built simulation facility. The intervention will be progressively rolled out across all 20 DHBs over a 5 year time frame, in four cohorts with five DHBs in each. Cohorts are matched for DHB size.
Cohort 1 training commenced on 3 March 2017. Cohort 2, 3, and 4 training will commence at 12 month intervals between 2018 and 2020. The intended start date for each new cohort is 1 February. In practice there may some delays in the start date in each cohort.
Delivery of the intervention is by trained, accredited instructors. The Project team will deliver the initial courses at each DHB and train local staff as instructors through a structured training programme with ongoing monitoring and support. Course quality will be monitored and assured through a central accreditation process, instructor networks and instructor exchange.
The NetworkZ team training days will be ongoing on a regular basis in each DHB, with an expectation that all OR staff will participate in a full course at least once every five years. We aim to train the majority of NZ OR staff during the course of this project and estimate that this will be over 6000 operating theatre staff.
The intervention will run from 3 March 2017, beginning in Cohort 1, and will continue until the training is running in all four Cohorts (i.e. all DHBs in New Zealand).
The pre-intervention period will be defined as the period of 12 months prior to the commencement of training in that Cohort. The implementation period will be 15 months following the commencement of training in that Cohort. The post-intervention period will be 12 months following the implementation period in that Cohort.
Our primary outcome measure, Days Alive and out of Hospital will be measure at 90 days post surgery for all patients undergoing surgery in New Zealand public hospitals. The measurement will be from 12 months prior to the intervention in Cohort 1 to 12 months after the implementation period in Cohort 4 (plus 90 days for surgical case data). Because of the 90 day post-surgery follow-up we will continue to collect data for each DHB for 90 days after the measurement period has ended
Intervention code [1] 296305 0
Other interventions
Comparator / control treatment
We are using a stepped wedge cluster design with staged roll-out of the intervention across the 20 DHBs in New Zealand in four cohorts at 12 month intervals, over a period of 4 years and including a transition phase (implementation period) of 15 months.
Using this design, the DHBs will act as their own controls as the intervention progressively rolls out across cohorts.
Control group
Active

Outcomes
Primary outcome [1] 300069 0
Days Alive and Out of Hospital at 90 Days. This data is downloaded from a national data base. All hospital admissions, discharges and readmissions are entered into the National Minimum Data Set (NMDS) as well as complete details of patient identity, hospital, diagnosis, patient co-morbidities, procedures undertaken. In addition, mortality is entered into another national database which can be linked to the NMDS. The files of all patients undergoing the surgical procedures of interest will be provided to the research team by the Ministry of Health, from which we will be able to calculate Days Alive and Out of Hospital.
Timepoint [1] 300069 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Primary outcome [2] 300225 0
Days out of hospital at 90 days post-surgery calculated from the NMDS.
Timepoint [2] 300225 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Primary outcome [3] 300226 0
Mortality and days alive post-surgery calculated from data from the Ministry of Health Mortality Data Set, linked through patient National Health Index, to the National Minimum Data Set.
Timepoint [3] 300226 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Secondary outcome [1] 328936 0
Post-operative complications recorded in the National Minimum Data Set
Timepoint [1] 328936 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Data on postoperative complications is entered by coders at DHB level into the NMDS. This will be measured from day 1 post surgery to 90 days post-surgery.
Secondary outcome [2] 328937 0
This is a composite secondary outcome of Accident Compensation Corporation (ACC) treatment injury claims including volume, cost, and categorisation from ACC claims data base.
Timepoint [2] 328937 0
Measured continuously from one year to the start of training on Cohort 1, to one year after the end of faculty assisted training in Cohort 4. This covers the period for all included claims from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Secondary outcome [3] 329078 0
Change in scores in a custom-designed teamwork processes survey.
Timepoint [3] 329078 0
A pre-measure in each cohort, and post-measure 12-15 months after the start of the intervention.
Secondary outcome [4] 329079 0
Change in safety culture score measured by the Surgical Safety Culture Survey
Timepoint [4] 329079 0
A pre-measure in each cohort in the period immediately before the intervention, and a post-measure about 12-15 months after.
Secondary outcome [5] 329080 0
Change in Quality of administration of the WHO Safe Surgery Checklist, measured by World Health Organisation Behaviourally Anchored Rating Scale (WHOBARS).
Timepoint [5] 329080 0
A pre-intervention observational measure in each cohort up to eight months before the start of training; and a post-intervention observational measure in each cohort about 12-15 months after the start of training in each DHB in the cohort.
Secondary outcome [6] 329086 0
DHB staff employment data on staff turnover, (numbers recruited, number of resignations or transfers).
Timepoint [6] 329086 0
Measured for the 12 month period prior to the start of training and the 12 months after the implementation period of the intervention in each DHB.
Secondary outcome [7] 329087 0
Organisational implementation process (qualitative exploration and understanding)
Timepoint [7] 329087 0
Data collected from each DHB during the period from five to twelve months after the start of training.
Secondary outcome [8] 329115 0
NetworkZ effectiveness and application to clinical practice (qualitative exploration and understanding)
Timepoint [8] 329115 0
OR staff interview data, collected from each DHB during the period from three to six months post the start of training.
Secondary outcome [9] 329116 0
Uptake within the DHBs measured as the number and proportion of relevant staff who have received the training (adjusted by the number of trained staff transferred in and out of the DHB).
Timepoint [9] 329116 0
From 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.
Secondary outcome [10] 330436 0
DHB staff records of sick-leave (count of days per staff member) by all operating room staff.
Timepoint [10] 330436 0
Measured for the 12 month period prior to the start of training and the 12 months after the implementation period of the intervention in each DHB.

Eligibility
Key inclusion criteria
Eligibility for the training and staff surveys and observations: All staff working in ORs to provide general, orthopaedic, urological, plastics, and ENT/ORL surgery in NZ. The main staff groups who will be included are: surgeons, surgical trainees, anaesthetists, anaesthetic trainees, anaesthetic technicians, anaesthetic technician trainees, and theatre nurses. We will also include some staff who have a managerial interest in the implementation of NetworkZ training in their DHB, including directors of surgery, directors of nursing, and quality managers.
Eligibility for patient outcome measure: all patients having surgical procedures with anaesthesia that are listed in the procedural codes of the Health Quality and Safety Commission (HQSC) Perioperative Mortality Review Committee (PMRC), involve the previously defined surgical disciplines, have an overnight stay (or die) in hospital, and are operated on in a public hospital operating room in New Zealand.
Minimum age
No limit
Maximum age
No limit
Gender
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
Exclusion criteria for training and evaluation: Some junior medical staff that have not commenced specialised training in anaesthesia or surgery, such as house officers, may be present in the teams that we observe and rate, but we will not seek to gather survey information from them in relation to our two staff safety culture surveys. The same will apply to Health Care Assistants, who may or may not be involved in OR work.
Exclusion criteria for surgical cases: minor procedures (i.e. patients who had their operation under local infiltration only).

Study design
Purpose of the study
Educational / counselling / training
Allocation to intervention
Randomised controlled trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
The allocation of DHBs to Cohorts was as follows: The DHBs were allocated to five groups according to population. The first Cohort of five DHBs was selected based on convenience and to ensure inclusion of DHBs of appropriate sizes and geographical distribution. The remaining DHBs were randomly allocated to Cohorts 2, 3 and 4 using a stratified allocation to ensure each Cohort contained the same distribution of large to small DHBs. Due to the pragmatic nature of quality improvement interventions, DHBs unable to participate in the training at their allocated time are able to swap with a DHB of a similar size in a subsequent Cohort.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Randomised number generator in Excel for DHB allocation.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Randomised number generator in Excel for DHB allocation
Masking / blinding
Open (masking not used)
Who is / are masked / blinded?



Intervention assignment
Other
Other design features
Stepped Wedge Cluster design
Phase
Not Applicable
Type of endpoint(s)
Safety/efficacy
Statistical methods / analysis
The Primary Outcome measure (DAOH at 90 days), will be calculated from the National Minimum Data Set and the Mortality Data set.
For all patients alive at 90 days, the DAOH score is 90 minus the number of days in hospital from the date of the surgical procedure. Any part of a day in hospital will count as one day. The maximum score for any patient included in the study is 89 because the day of the procedure will count as one day in hospital. A death prior to leaving hospital will count as zero. A death within the 90 day period when the patient was discharged home from hospital will count as the number of days out of hospital during the 90 day period.
A second operation within the 90 day period will be treated as the same event with the exception of any situation where the first operation is a minor diagnostic procedure leading directly to the definitive procedure (e.g. biopsy under GA confirming Ca, followed by definitive surgery within the 90 day period).
We will include only patients in the surgical disciplines receiving the intervention, patients requiring an overnight admission, day-stay patients who die on the day of surgery, patients undergoing regional or general anaesthesia, and patients receiving an intervention listed as relevant to the concerns of the HQSC’s PMRC.

The study design will be the transition form of the stepped wedge cluster design [1}, where the transition phase will be a 15-month period of implementation of the intervention following the start of training. The total period of data collection for the DAOH analysis will be from 12 months prior to the first training day in the first DHB in Cohort 1 until 12 months after the end of the implementation period in Cohort 4. Data from the implementation (transition) phase will not be used in the analysis given that it cannot be assigned to either intervention or control arms.
Patients’ mean DAOH scores will be classified into pre-intervention (control) and post-intervention (intervention) for analysis.
We will use general linear modelling to estimate the effect of the intervention on DAOH.
Information on gender, age, surgical procedure, ASA status will also be collected and factored in to the analysis.
Year number post intervention will be included in the analysis to gauge information about the effect over time (attenuation, continuance or improvement). We will also include time (year) in the analysis to deal with any confounding from other temporal changes over time.

We will conduct a similar comparative analysis for Days Out of Hospital (for survivors); for post-surgery mortality; a defined set of post-surgery complications (for example wound infection, renal impairment, thrombotic event).
We will use a similar approach to the analysis of the ACC claims data base for treatment injury claims (number and costs).

[1] Hemming, K., et al., The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ : British Medical Journal, 2015. 350.

The SSCS survey will be non-identifiable, but (in order to identify a group of repeat surveys) we will ask all participants completing the post-intervention measure if they completed the pre-measure 12 months prior. Pre-post survey responses will be compared, with the subset of those who completed both surveys used to help establish comparability of pre-post responding groups.

The teamwork perceptions surveys will be provided to participants with identifiable numbers, which will enable us to conduct paired t-tests of pre- and post-intervention survey responses from the same individuals.

For both the SSCS and teamwork perceptions surveys we will establish the "dose effect" of intervention by gathering and analysing data on which staff have received the training.

For each cohort we will analyse observational data collected via the WHOBARS tool. We will then conduct pre- and post-intervention t-tests on the WHOBARS scores. Because our observational and training records will enable us to identify the proportion of observed staff, and OR staff in each DHB, who have been trained. We will be able to investigate the relationship between these variables (i.e. the dose) and WHOBARS scores (the effect).

Quantitative measures will be compared between pre- and post-rollout phases.

Qualitative data from the interviews will undergo framework analysis to identify barriers and enablers to the implementation of NetworkZ training by DHBs, and improved communication by OR staff. The framework for this analysis will be the Organising for Quality model [2]. The themes identified will be analysed at the level of entire cohorts.

[2] Bate, P,. et al., Organising for Quality: The Improvement Journeys of Leading Hospitals in Europe and the United States. Oxford & New York, Radcliffe Publishing, 2008.

Recruitment
Recruitment status
Recruiting
Date of first participant enrolment
Anticipated
Actual
Date of last participant enrolment
Anticipated
Actual
Date of last data collection
Anticipated
Actual
Sample size
Target
Accrual to date
Final
Recruitment outside Australia
Country [1] 8368 0
New Zealand
State/province [1] 8368 0
nationwide

Funding & Sponsors
Funding source category [1] 294871 0
Government body
Name [1] 294871 0
Accident Compensation Corporation
Address [1] 294871 0
PO Box 242
Wellington 6140
Country [1] 294871 0
New Zealand
Funding source category [2] 298769 0
Government body
Name [2] 298769 0
Lotteries Health Research Fund New Zealand
Address [2] 298769 0
The Department of Internal Affairs
Community Operations 147 Lambton Quay
Wellington
Post code 6011
Country [2] 298769 0
New Zealand
Funding source category [3] 299668 0
Other Collaborative groups
Name [3] 299668 0
Australia and New Zealand College of Anaesthetists
Address [3] 299668 0
PO Box 6095,
Melbourne,
Victoria 3004,
Country [3] 299668 0
Australia
Primary sponsor type
Individual
Name
Professor Jennifer Weller
Address
Centre for Medical and Health Sciences Education
School of Medicine
The University of Auckland
Private Bag 92019
Auckland 1142

Country
New Zealand
Secondary sponsor category [1] 293752 0
None
Name [1] 293752 0
Address [1] 293752 0
Country [1] 293752 0

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 296252 0
Health and Disability Ethics Committees
Ethics committee address [1] 296252 0
Ministry of Health
PO Box 5013
Wellington 6011
Ethics committee country [1] 296252 0
New Zealand
Date submitted for ethics approval [1] 296252 0
15/08/2016
Approval date [1] 296252 0
03/10/2016
Ethics approval number [1] 296252 0
16/NTB/143

Summary
Brief summary
This project evaluates the effectiveness of the NetworkZ training intervention for operating room (OR) staff in all DHBs in New Zealand (NZ). The aim of the intervention is to improve outcomes for surgical patients, and improve teamwork in the OR.
Our primary hypothesis is that NetworkZ will lead to improved scores for Days Alive and out of Hospital (DAOH) at 90 days. The NetworkZ training is for staff in general, orthopaedic, urological, plastics, and ENT/ORL surgery in all DHBs in New Zealand. Each training session will be provided by accredited instructors to multidisciplinary teams of about 10 staff. The training comprises realistic simulations of challenging surgical cases, debriefs of the cases, and discussions of specific teamwork and communication strategies. The training focusses on shared mental models, structured communication, and mutual respect and trust.
The NetworkZ training will be rolled out through all NZ DHBs in four cohorts of five DHBs over a period of five years. We will conduct a concurrent evaluation throughout using a stepped wedge cluster research design. This design will allow us to compare pre- and post-intervention cohorts, and enhance our capacity to separate training effects from long term trends and changes in the DHBs.
Staff interviews will seek to identify factors that assist or inhibit the implementation of NetworkZ training in the different DHBs.
Primary Outcome Measure:
Days Alive and Out of Hospital (at 90 days). This looks at patients who are alive AND the number of days they are out of hospital, in the 90 days following surgery. We will also analyse mortality, and days out of hospital separately. The data for this analysis will be drawn from the NMDS.
Secondary Outcome Measures:
Post-surgical complications from the NMDS.Number and costs of post-surgical treatment injury claims, from ACC data base.
Safety Climate pre-post intervention survey using the Surgical Culture Safety Survey developed and validated by the Harvard School of Public Health.
Teamwork perceptions pre-post intervention survey, using a tool we have custom-designed to measure NetworkZ training processes.
The quality of implementation of the Safe Surgery Checklist pre-post intervention, measured by the WHOBARS observation instrument, which we have previously developed and validated.
The number and proportion of OR staff trained in the DHBs.
Staff retention and the use of sick leave from DHB records.
Interview data exploring the organisational implementation process in the DHBs and how staff apply the training in their OR work. (Framework Analysis).
Trial website
www.networkz.ac.nz
Trial related presentations / publications
Public notes
Publications arising from the NetworkZ pilot study of 20 courses in two DHBs are listed below (Note: The NetworkZ programme was originally known as MORSim (Multidisciplinary Operating Room Simulation)),
1. Weller JM, Civil I, Torrie J, Cumin D, Garden A, Corter A, Merry AF. Can team training make surgery safer? Lessons for national implementation of a simulation-based programme. New Zealand Medical Journal 2016. October 14th. Volume 129 Number 1443
2. Weller J et al. Improved scores for observed teamwork in the clinical environment following a multidisciplinary operating room simulation intervention. New Zealand Medical Journal 5 August 2016, Vol 129 No 1439 ISSN 1175-8716
3. Cumin, D., Skilton, C., & Weller, J. (2016). Information transfer in multidisciplinary operating room teams: a simulation-based observational study. BMJ Quality & Safety. 2016. doi:10.1136/bmjqs-2015-005130
4. Weller, J., Cumin, D., Torrie, J., Boyd, M., Civil, I., Madell, D., . . . Merry, A. F. (2015). Multidisciplinary operating room simulation-based team training to reduce treatment errors: a feasibility study in New Zealand hospitals. The New Zealand Medical Journal, 128(1418), 40-51.
5. Nakarada-Kordic, I., Weller, J., Webster, C., Cumin, D., Frampton, C., Boyd, M., Merry, A.F. Assessing the similarity of mental models in operating room team members and implications for patient safety: a prospective, replicated study. BMC Medical Education 2016. 16 (1), 229
Attachments [2] 2445 2445 0 0
Attachments [3] 2446 2446 0 0

Contacts
Principal investigator
Name 70122 0
Prof Jennifer Weller
Address 70122 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70122 0
New Zealand
Phone 70122 0
+64 9 923 9459
Fax 70122 0
+64 9 373 7641
Email 70122 0
j.weller@auckland.ac.nz
Contact person for public queries
Name 70123 0
Prof Jennifer Weller
Address 70123 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70123 0
New Zealand
Phone 70123 0
+64 9 923 9459
Fax 70123 0
+64 9 373 7641
Email 70123 0
j.weller@auckland.ac.nz
Contact person for scientific queries
Name 70124 0
Prof Jennifer Weller
Address 70124 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70124 0
New Zealand
Phone 70124 0
+64 9 923 9459
Fax 70124 0
+64 9 373 7641
Email 70124 0
j.weller@auckland.ac.nz

No data has been provided for results reporting
Summary results
Not applicable