1 Prepare R environment

Cleanup

##           used (Mb) gc trigger (Mb) max used (Mb)
## Ncells  548707 29.4    1224502 65.4   686462 36.7
## Vcells 1037568  8.0    8388608 64.0  1875970 14.4
## Warning: package 'plyr' was built under R version 4.4.1
## Warning: package 'dplyr' was built under R version 4.4.1
## Warning: package 'tidyr' was built under R version 4.4.1
## Warning: package 'reshape2' was built under R version 4.4.1
## Warning: package 'lubridate' was built under R version 4.4.1
## Warning: package 'purrr' was built under R version 4.4.1
## Warning: package 'Hmisc' was built under R version 4.4.1
## Warning: package 'stringr' was built under R version 4.4.1
## Warning: package 'psych' was built under R version 4.4.1
## Warning: package 'exact2x2' was built under R version 4.4.1
## Warning: package 'exactci' was built under R version 4.4.1
## Warning: package 'testthat' was built under R version 4.4.1
## Warning: package 'car' was built under R version 4.4.1
## Warning: package 'carData' was built under R version 4.4.1
## Warning: package 'lme4' was built under R version 4.4.1
## Warning: package 'pscl' was built under R version 4.4.1
## Warning: package 'betareg' was built under R version 4.4.1
## Warning: package 'glmmTMB' was built under R version 4.4.1
## Warning: package 'arm' was built under R version 4.4.1
## Warning: package 'rcompanion' was built under R version 4.4.1
## Warning: package 'PMCMRplus' was built under R version 4.4.1
## Warning: package 'lmtest' was built under R version 4.4.1
## Warning: package 'zoo' was built under R version 4.4.1
## Warning: package 'DHARMa' was built under R version 4.4.1
## Warning: package 'epitab' was built under R version 4.4.1
## Warning: package 'xtable' was built under R version 4.4.1
## Warning: package 'kableExtra' was built under R version 4.4.1
## Warning: package 'sjPlot' was built under R version 4.4.1
## Warning: package 'ggplot2' was built under R version 4.4.1
## Warning: package 'ggthemes' was built under R version 4.4.1
## Warning: package 'GGally' was built under R version 4.4.1
## Warning: package 'corrplot' was built under R version 4.4.1
## Warning: package 'namer' was built under R version 4.4.1

Set working directory

Load user-defined functions

## Loading required package: tmvtnorm
## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'tmvtnorm'

Import files

##   [1] "prolific.id"         "participant.id"      "participant.status" 
##   [4] "cost"                "arousal1"            "arousal2"           
##   [7] "arousal3"            "attentionchckyes"    "attentionoutcomes"  
##  [10] "bath"                "birthyear"           "comment"            
##  [13] "confirmconsent"      "consent"             "detect4"            
##  [16] "detect5"             "detect1"             "detect2"            
##  [19] "detect3"             "detectiontext"       "expbathing"         
##  [22] "expcost"             "expekf"              "expfloodh"          
##  [25] "expfloodr"           "expprof"             "expprof.text"       
##  [28] "expprofyears"        "gender"              "valence1"           
##  [31] "valence2"            "valence3"            "ekf"                
##  [34] "uk"                  "other.country"       "floodr"             
##  [37] "floodh"              "sewerservice"        "tot.affect1"        
##  [40] "tot.affect2"         "tot.affect3"         "tot.debriefing"     
##  [43] "tot.demographics"    "tot.end"             "tot.experience"     
##  [46] "tot.intro"           "tot.outcomes"        "glt"                
##  [49] "crt"                 "bnt"                 "e.cond"             
##  [52] "practice.1"          "practice.2"          "practice.3"         
##  [55] "rpe_1_1"             "rpe_1_2"             "rpe_2_1"            
##  [58] "rpe_2_2"             "rpe_5_1"             "rpe_5_2"            
##  [61] "rpe_6_1"             "rpe_6_2"             "rpe_9_1"            
##  [64] "rpe_9_2"             "rpe_10_1"            "rpe_10_2"           
##  [67] "rpe_11_1"            "rpe_11_2"            "rpe_12_1"           
##  [70] "rpe_12_2"            "rpe_13_1"            "rpe_13_2"           
##  [73] "rpe_14_1"            "rpe_14_2"            "rpe_15_1"           
##  [76] "rpe_15_2"            "rpe_17_1"            "rpe_17_2"           
##  [79] "rpe_20_1"            "rpe_20_2"            "rpe_21_1"           
##  [82] "rpe_21_2"            "rpe_22_1"            "rpe_22_2"           
##  [85] "rpe_23_1"            "rpe_23_2"            "rpe_24_1"           
##  [88] "rpe_24_2"            "rpe_25_1"            "rpe_25_2"           
##  [91] "rpe_26_1"            "rpe_26_2"            "rpe_27_1"           
##  [94] "rpe_27_2"            "rpv_1_1"             "rpv_1_2"            
##  [97] "rpv_2_1"             "rpv_2_2"             "rpv_5_1"            
## [100] "rpv_5_2"             "rpv_6_1"             "rpv_6_2"            
## [103] "rpv_9_1"             "rpv_9_2"             "rpv_10_1"           
## [106] "rpv_10_2"            "rpv_11_1"            "rpv_11_2"           
## [109] "rpv_12_1"            "rpv_12_2"            "rpv_13_1"           
## [112] "rpv_13_2"            "rpv_14_1"            "rpv_14_2"           
## [115] "rpv_15_1"            "rpv_15_2"            "rpv_17_1"           
## [118] "rpv_17_2"            "rpv_20_1"            "rpv_20_2"           
## [121] "rpv_21_1"            "rpv_21_2"            "rpv_22_1"           
## [124] "rpv_22_2"            "rpv_23_1"            "rpv_23_2"           
## [127] "rpv_24_1"            "rpv_24_2"            "rpv_25_1"           
## [130] "rpv_25_2"            "rpv_26_1"            "rpv_26_2"           
## [133] "rpv_27_1"            "rpv_27_2"            "ct_1_crt"           
## [136] "ct_2_crt"            "ct_3_crt"            "ct_1_bnt"           
## [139] "ct_2a_bnt"           "ct_2b_bnt"           "ct_3_bnt"           
## [142] "ct_1_glt"            "ct_2_glt"            "rpe.why_1_1"        
## [145] "rpe.why_2_1"         "rpe.why_5_1"         "rpe.why_6_1"        
## [148] "rpe.why_9_1"         "rpe.why_10_1"        "rpe.why_11_1"       
## [151] "rpe.why_12_1"        "rpe.why_13_1"        "rpe.why_14_1"       
## [154] "rpe.why_15_1"        "rpe.why_17_1"        "rpe.why_20_1"       
## [157] "rpe.why_21_1"        "rpe.why_22_1"        "rpe.why_23_1"       
## [160] "rpe.why_24_1"        "rpe.why_25_1"        "rpe.why_26_1"       
## [163] "rpe.why_27_1"        "rpe.why_1_2"         "rpe.why_2_2"        
## [166] "rpe.why_5_2"         "rpe.why_6_2"         "rpe.why_9_2"        
## [169] "rpe.why_10_2"        "rpe.why_11_2"        "rpe.why_12_2"       
## [172] "rpe.why_13_2"        "rpe.why_14_2"        "rpe.why_15_2"       
## [175] "rpe.why_17_2"        "rpe.why_20_2"        "rpe.why_21_2"       
## [178] "rpe.why_22_2"        "rpe.why_23_2"        "rpe.why_24_2"       
## [181] "rpe.why_25_2"        "rpe.why_26_2"        "rpe.why_27_2"       
## [184] "dom.2"               "dom2.rank"           "risk.e_1"           
## [187] "risk.e_2"            "risk.v_1"            "risk.v_2"           
## [190] "risk.g5.e_1"         "risk.g5.e_2"         "risk.g5.v_1"        
## [193] "risk.g5.v_2"         "risk.g9.e_1"         "risk.g9.e_2"        
## [196] "risk.g9.v_1"         "risk.g9.v_2"         "risk.g10.e_1"       
## [199] "risk.g10.e_2"        "risk.g10.v_1"        "risk.g10.v_2"       
## [202] "risk.g21.e_1"        "risk.g21.e_2"        "risk.g21.v_1"       
## [205] "risk.g21.v_2"        "risk.e"              "risk.v"             
## [208] "risk.diff"           "risk.m.diff"         "risk.nm.e"          
## [211] "risk.nm.v"           "risk.difficult.diff" "risk.easy.diff"     
## [214] "exp"                 "training"            "tot.overall"        
## [217] "s.per.choice"        "s.per.choice.t1"     "s.per.choice.t2"    
## [220] "s.per.choice.cost"   "s.per.choice.dom.2"  "indiff"             
## [223] "e.gain"              "e.loss"              "p.gain"             
## [226] "p.loss"              "etc"                 "detected"           
## [229] "detected.m"          "detected.5_1"        "detected.9_1"       
## [232] "detected.10_1"       "detected.21_1"       "detected.5_2"       
## [235] "detected.9_2"        "detected.10_2"       "detected.21_2"      
## [238] "detected.difficult"  "detected.easy"       "detect.post"        
## [241] "detected.post"       "smc_1"               "smc_2"              
## [244] "smc"                 "phi_1"               "phi_2"              
## [247] "phi"                 "smc_m_1"             "smc_m_2"            
## [250] "smc_m"               "phi_m_1"             "phi_m_2"            
## [253] "phi_m"               "smc_nm_1"            "smc_nm_2"           
## [256] "smc_nm"              "phi_nm_1"            "phi_nm_2"           
## [259] "phi_nm"              "smc_m_difficult"     "smc_m_easy"         
## [262] "risk"

2 Descriptive statistics of dependent and explanatory variables

2.1 Risk preference stability

2.1.1 Risk preference stability overall

(1) How many shift and by how much?

Note: Negative values are change towards risk seeking, positive values towards risk averse

## 
## -22 -18 -16 -14 -12 -10  -8  -6  -4  -2   0   2   4   6   8  10  12  14  16  18 
##   2   2   2   8   8  15  13  21  32  32  31  29  21  26  19  13  12   4   5   1 
##  20  22 
##   1   1
##               Risk attitude   n min Q.25 median        mean Q.75 max       sd
## 1          1st trial series 298 -20   -6      0 -0.24161074    6  18 7.669653
## 2          2nd trial series 298 -20   -6      0 -0.14765101    6  18 7.881595
## 3      Difference (1st-2nd) 298 -22   -4      0  0.09395973    6  22 7.671138
## 4 abs(Difference (1st-2nd)) 298   0    2      6  6.08053691    8  22 4.664526
##   kurtosis    skewness
## 1 2.385774 -0.01546359
## 2 2.228878 -0.05171120
## 3 2.925944 -0.02808264
## 4 3.551229  0.90198936
Table. Risk attitude scores between 1st and 2nd trial series
Risk attitude Condition risk seeking risk neutral risk averse n min Q.25 median mean Q.75 max sd kurtosis skewness
All conditions
All 1st trial series 141 24 133 298 -20 -6 0 -0.242 6 18 7.670 2.386 -0.015
All 2nd trial series 134 28 136 298 -20 -6 0 -0.148 6 18 7.882 2.229 -0.052
All Difference (1st-2nd) 135 31 132 298 -22 -4 0 0.094 6 22 7.671 2.926 -0.028
All Difference (absolute no.) – – – 298 0 2 6 6.081 8 22 4.665 3.551 0.902
By experimental condition
All Difference (numeric) 66 15 70 151 -18 -4 0 0.848 6 22 7.816 2.794 0.094
All Difference (icons) 69 16 62 147 -22 -6 0 -0.680 4 16 7.467 2.943 -0.202

(2) Get overview of number of risk seeking (negative), risk neutral (zero), and risk averse (positive) attitudes

Table. Risk attitude scores across two series of elicitation trials
Set Condition Risk seeking Risk neutral Risk averse n min Q.25 median mean Q.75 max sd kurtosis skewness
All outcome domains
All both 138 15 145 298 -32 -12 0 -0.389 10 28 13.529 2.172 -0.025
All numeric 21 7 123 151 -26 3 10 8.530 14 28 9.927 3.495 -0.572
All icons 117 8 22 147 -32 -16 -10 -9.551 -2 28 10.214 3.936 0.609
Common domain
Cost both 135 19 144 298 -18 -6 0 -0.181 6 20 7.851 2.394 -0.176
Cost numeric 26 10 115 151 -16 2 4 4.609 8 20 5.699 3.441 -0.353
Cost icons 109 9 29 147 -18 -10 -6 -5.102 0 14 6.626 2.751 0.238
Priority domain
Other both 139 31 128 298 -20 -6 0 -0.208 6 18 7.532 2.379 0.017
Other numeric 32 18 101 151 -14 0 4 3.921 8 18 6.337 2.910 -0.299
Other icons 107 13 27 147 -20 -10 -4 -4.449 0 14 6.197 2.931 0.313
By priority domain
House flooding both 64 18 70 152 -16 -4 0 0.618 6 18 7.481 2.565 -0.014
Waterbody pollution both 46 9 40 95 -20 -8 0 -0.968 6 14 7.816 2.072 -0.058
Road flooding both 12 3 13 28 -10 -4 0 1.071 6 16 6.960 2.204 0.243
Bathing restriction both 17 1 5 23 -14 -8 -6 -4.087 -1 8 6.014 2.325 0.237
By trial series
1st trial series both 141 24 133 298 -20 -6 0 -0.242 6 18 7.670 2.386 -0.015
2nd trial series both 134 28 136 298 -20 -6 0 -0.148 6 18 7.882 2.229 -0.052
Change between trials
Change (1st-2nd) both 135 31 132 298 -22 -4 0 0.094 6 22 7.671 2.926 -0.028
Change (1st-2nd) numeric 66 15 70 151 -18 -4 0 0.848 6 22 7.816 2.794 0.094
Change (1st-2nd) icons 69 16 62 147 -22 -6 0 -0.680 4 16 7.467 2.943 -0.202
No. of changes both – – – 298 0 2 6 6.081 8 22 4.665 3.551 0.902

Main observations:

  • With a median of -0.389 and mean of -0.389 (sd= 13.529), risk attitudes are counterbalanced and vary strongly
  • The experimental condition seems to strongly influence risk attitudes
    • in the numeric condition, risk averse (81.46%), risk seeking (13.91%), neutral (4.64%)
    • in the icon condition, risk averse (14.97%), risk seeking (79.59%), neutral (5.44%)
    • effect apparent across domains
  • Small difference between trial series
  • Change between trials on average almost neutral, albeit large variability (mu= 0.094, sd= 7.671)
    • in numeric condition more risk averse (mu= 0.848, sd= 7.816)
    • in icon condition more risk seeking (mu= -0.68, sd= 7.467)
    • overall number of changes on gambles (mu= 6.081, sd= 4.665)
  • ADD?! risk attitude score from one type to another?

2.1.2 Risk preference stability on manipulated trials

(1) How many shift, by how much?

Table. No of choices changed on manipulated gambles
Changes Participants
0 14
1 31
2 89
3 66
4 53
5 25
6 17
7 3
Table. Summary of choices changed on manipulated gambles
n min Q.25 median mean Q.75 max sd kurtosis skewness
298 0 2 3 2.909 4 7 1.525 2.762 0.359

(2) Any difference on detected-M or nondetected-M trials?

Table. Changed choices on detected and non-detected manipulated gambles
unchanged changed
undetected 1369 795
detected 148 72 
##         difference
## detected  0  1  2  3  4  5  6  7
##        0 12 19 68 48 41 22 11  2
##        1  1  3 10  6  5  2  4  0
##        2  0  1  3  3  2  0  0  0
##        3  1  1  1  3  2  1  0  0
##        4  0  1  3  1  3  0  2  1
##        5  0  1  1  0  0  0  0  0
##        6  0  1  2  2  0  0  0  0
##        7  0  2  1  1  0  0  0  0
##        8  0  2  0  2  0  0  0  0
##         difference
## detected  0  2  4  6  8 10 12 14 16 18 20 22
##        0 24 46 36 36 24 23 13  8  6  3  1  3
##        1  3  6 10  2  3  3  2  1  1  0  0  0
##        2  1  3  3  1  0  0  0  1  0  0  0  0
##        3  0  4  1  0  1  1  0  2  0  0  0  0
##        4  1  0  0  3  3  0  4  0  0  0  0  0
##        5  1  0  0  0  0  1  0  0  0  0  0  0
##        6  0  1  0  3  1  0  0  0  0  0  0  0
##        7  1  0  2  1  0  0  0  0  0  0  0  0
##        8  0  1  1  1  0  0  1  0  0  0  0  0
##   n min Q.25 median     mean Q.75 max       sd kurtosis  skewness
##  75   0    2      3 2.853333    4   7 1.556966 2.921831 0.6133666
##    n min Q.25 median     mean Q.75 max       sd kurtosis  skewness
##  223   0    2      3 2.928251    4   7 1.516949 2.720678 0.2690666
##   n min Q.25 median     mean Q.75 max       sd kurtosis  skewness
##  75   0    2      4 5.786667    8  16 4.153453 2.411086 0.5987997
##    n min Q.25 median     mean Q.75 max       sd kurtosis  skewness
##  223   0    2      6 6.179372   10  22 4.828978 3.650087 0.9454074
##   n min  Q.25 median      mean   Q.75  max        sd kurtosis  skewness
##  75 0.4 0.575   0.65 0.6696667 0.7625 0.95 0.1216645 2.410324 0.3097695
##    n min  Q.25 median      mean  Q.75   max        sd kurtosis   skewness
##  223 0.3 0.575   0.65 0.6449552 0.725 0.925 0.1205038 2.930136 -0.1443457

(3) Risk attitude stability and SMC by outcome domain

##    n min Q.25 median     mean Q.75 max       sd kurtosis skewness
##  298   0    2      4 4.127517    6  16 3.367079  3.79573 1.010676
##    n min Q.25 median     mean Q.75 max       sd kurtosis  skewness
##  298   0    2      4 3.926174    6  18 3.145397 4.108992 0.8673504
##    n  min Q.25 median      mean Q.75 max        sd kurtosis   skewness
##  298 0.15 0.55    0.7 0.6659396 0.75   1 0.1536513 3.203327 -0.4573215
##    n min Q.25 median      mean Q.75  max        sd kurtosis    skewness
##  298 0.2 0.55  0.625 0.6364094 0.75 0.95 0.1533296 2.605394 -0.05397495

Shapiro Wilk Normality test on risk attitude scores with H0 that data follow normal distribution * H0: rejected

Shapiro Wilk Normality test on change in risk attitude scores with H0 that data follow normal distribution * H0: not rejected

2.2 Choice blindness manipulation detection (CBMD)

2.2.1 Concurrent detections

(1) Altogether

Table. Reasoning for choice by gamble with n= 298 participants. Bold indicates manipulated gambles
Gamble P(A) Choice correction Indifferent Expected gain Expected loss Possible gain Possible loss Other reason
Common domain (cost)
G1 0.671 3 12 117 18 100 40 8
G2 0.721 2 11 142 16 93 26 8
G5 0.503 22 17 106 18 89 41 5
G6 0.423 6 16 52 83 73 60 8
G9 0.436 25 9 87 12 116 43 6
G10 0.366 31 23 52 66 60 60 6
G11 0.617 1 15 66 64 65 75 12
G12 0.440 1 20 22 131 21 97 6
G13 0.537 4 10 24 139 21 95 5
G14 0.487 5 27 51 71 67 67 10
G15 0.201 2 23 25 142 13 84 9
G17 0.473 3 25 13 113 23 116 5
G20 0.470 2 12 45 67 80 75 17
G21 0.500 29 6 94 24 105 36 4
G22 0.329 5 18 88 49 77 51 10
G23 0.554 3 11 110 17 111 38 8
G24 0.520 1 5 110 16 113 41 12
G25 0.416 2 6 27 114 44 94 11
G26 0.430 4 17 80 69 48 69 11
G27 0.685 2 10 61 73 71 71 10
Priority domain
G1 0.534 2 31 55 56 61 77 16
G2 0.557 1 17 72 42 90 69 7
G5 0.440 28 28 68 37 64 67 6
G6 0.433 2 22 55 71 49 90 9
G9 0.473 29 24 50 47 57 83 8
G10 0.426 24 20 36 65 60 82 11
G11 0.393 1 19 37 85 52 93 11
G12 0.433 2 24 25 93 28 116 10
G13 0.423 4 15 19 95 35 121 9
G14 0.443 3 35 40 59 52 101 8
G15 0.265 3 22 25 102 22 114 10
G17 0.574 0 29 26 87 26 122 8
G20 0.537 6 24 37 69 48 102 12
G21 0.624 32 15 62 46 56 79 8
G22 0.453 5 26 71 60 57 73 6
G23 0.507 2 26 64 52 65 81 8
G24 0.376 3 8 75 38 100 66 8
G25 0.389 1 12 28 86 46 114 11
G26 0.493 2 28 48 81 38 91 10
G27 0.557 5 14 41 71 63 95 9
Table. Amount of manipulated gambles detected
Corrections Participants
0 223
1 31
2 9
3 9
4 11
5 2
6 5
7 4
8 4
Summary of manipulation detection
Participants n min Q.25 median mean Q.75 max sd kurtosis skewness
all 298 0 0 0 0.738 0.75 8 1.685 9.772 2.686
detection > 0 75 1 1 2 2.933 4.00 8 2.208 2.688 0.927
Table. Choice blindness detection of manipulated gambles by outcome domain with n= 298 participants
Set Condition Detection Gamble 5 Gamble 9 Gamble 10 Gamble 21
All outcome domains
All both 220 50 54 55 61
All icons 129 30 29 33 37
All numeric 91 20 25 22 24
Common domain
Cost both 107 22 25 31 29
Cost icons 61 12 15 17 17
Cost numeric 46 10 10 14 12
Priority domain
Other both 113 28 29 24 32
Other icons 68 18 14 16 20
Other numeric 45 10 15 8 12
By priority domain
House flooding icons 45 11 8 10 16
House flooding numeric 20 4 8 3 5
Waterbody pollution icons 15 5 4 3 3
Waterbody pollution numeric 14 4 4 3 3
Road flooding icons 3 1 1 1 0
Road flooding numeric 4 1 1 0 2
Bathing restriction icons 5 1 1 2 1
Bathing restriction numeric 7 1 2 2 2
  • Proportion tables NEEDS CORRECTION AND MAKING PROPER TABLE 
##   e.cond  dom.2 detection gamble 5 gamble 9 gamble 10 gamble 21    cost  cost 5
## 1   icon   bath   0.14773  0.13636  0.13636   0.18182   0.13636 0.18182 0.18182
## 2   icon    ekf   0.10556  0.08889  0.11111   0.11111   0.11111 0.12778 0.06667
## 3   icon floodh   0.11058  0.10897  0.08974   0.10256   0.14103 0.07692 0.07692
## 4   icon floodr   0.08654  0.07692  0.07692   0.11538   0.07692 0.11538 0.07692
## 5   tree   bath   0.12500  0.12500  0.12500   0.16667   0.08333 0.10417 0.16667
## 6   tree    ekf   0.07750  0.08000  0.09000   0.07000   0.07000 0.08500 0.08000
## 7   tree floodh   0.06419  0.04730  0.06757   0.06081   0.08108 0.06081 0.04054
## 8   tree floodr   0.08333  0.06667  0.10000   0.06667   0.10000 0.10000 0.06667
##    cost 9 cost 10 cost 21   other other 5 other 9 other 10 other 21
## 1 0.18182 0.18182 0.18182 0.11364 0.09091 0.09091  0.18182  0.09091
## 2 0.13333 0.15556 0.15556 0.08333 0.11111 0.08889  0.06667  0.06667
## 3 0.07692 0.07692 0.07692 0.14423 0.14103 0.10256  0.12821  0.20513
## 4 0.07692 0.15385 0.15385 0.05769 0.07692 0.07692  0.07692  0.00000
## 5 0.08333 0.16667 0.00000 0.14583 0.08333 0.16667  0.16667  0.16667
## 6 0.10000 0.08000 0.08000 0.07000 0.08000 0.08000  0.06000  0.06000
## 7 0.02703 0.08108 0.09459 0.06757 0.05405 0.10811  0.04054  0.06757
## 8 0.13333 0.13333 0.06667 0.06667 0.06667 0.06667  0.00000  0.13333

(2) Detections by outcome domain

##    n min Q.25 median     mean Q.75 max       sd kurtosis skewness
##  298   0    0      0 0.738255 0.75   8 1.685334 9.771502 2.685596
##    n min Q.25 median      mean Q.75 max        sd kurtosis skewness
##  298   0    0      0 0.3590604    0   4 0.9366833 9.954467 2.800953
##    n min Q.25 median      mean Q.75 max        sd kurtosis skewness
##  298   0    0      0 0.3791946    0   4 0.9140707 9.856169 2.731148
##    n min Q.25 median      mean Q.75 max        sd kurtosis skewness
##  298   0    0      0 0.2516779 0.75   1 0.4347071 2.309656 1.144402
##    n min Q.25 median      mean Q.75 max       sd kurtosis skewness
##  298   0    0      0 0.1677852    0   1 0.374304 4.161613 1.778092
##    n min Q.25 median      mean Q.75 max        sd kurtosis skewness
##  298   0    0      0 0.2013423    0   1 0.4016777 3.218768 1.489553

(3) Detections by stimuli type / experimental condition

2.2.2 Ex-post detection

Questions were

  1. Did you notice anything unusual during this survey?
  2. Did you suspect that anything was amiss with your selected option?
  3. Did you suspect any trickery on behalf of the program leading you through the questions?
  4. Do you believe you were in the first group?
  5. Did you notice that your answers were swapped?

Altogether

##   detect1 detect2 detect3 detect4 detect5
## 0     243     236     233     198     223
## 1      55      62      65     100      75

Before suggestion (Q1-3 only)

## 
##   0   1   2   3 
## 207  36  19  36

After suggestion (Q4-5 only)

## 
##   0   1   2 
## 196  29  73

Number of detections per participants (across questions Q1-5)

## 
##   0   1   2   3   4   5 
## 164  44  27  20  16  27

2.3 Predictors of CBMD and rank stability

2.3.1 Domain rank

Table. Importance ranking of outcome domains
outcome n min Q.25 median mean Q.75 max sd kurtosis skewness
House flooding 298 1 1 1 1.973 3 5 1.209 2.613 0.957
Waterbody pollution 298 1 1 2 2.326 3 5 1.183 2.372 0.573
Road flooding 298 1 2 3 3.034 4 5 1.100 2.288 0.010
Cost 298 1 2 3 3.275 4 5 1.227 2.132 -0.284
Bathing restriction 298 1 4 5 4.393 5 5 0.959 4.405 -1.518
Table. Importance ranking by domain (rank frequency)
Cost House flooding Waterbody pollution Road flooding Bathing restriction
30 151 90 24 3
51 65 91 74 17
77 30 62 97 32
87 43 40 74 54
53 9 15 29 192
Table. Respondents per domain
Bathing restriction Waterbody pollution House flooding Road flooding Sum
icon 11 45 78 13 147
tree 12 50 74 15 151
Sum 23 95 152 28 298

2.3.2 Cognitive abilty scores

Note: BNT has been rescaled from 1-4 to 0-3 to be similarly scaled to CRT and GLT tasks

Table. Performance on cognitive tasks
Test Condition n min Q.25 median mean Q.75 max sd kurtosis skewness
GLT both 298 0 1 2 1.517 2 2 0.552 2.198 -0.548
BNT both 298 0 0 1 1.164 2 3 1.071 2.070 0.558
CRT both 298 0 0 1 1.275 2 3 1.188 1.549 0.276

2.3.3 Experience with urban drainage service outcomes

Table. Summary of service outcomes experienced
experience n min Q.25 median mean Q.75 max sd kurtosis skewness
House flooding 298 0 0 0 0.128 0 1 0.334 5.988 2.233
Waterbody pollution 298 0 0 0 0.171 0 1 0.377 4.050 1.746
Road flooding 298 0 0 1 0.664 1 1 0.473 1.485 -0.696
Cost 298 0 0 0 0.342 1 1 0.475 1.442 0.665
Bathing 298 0 0 0 0.265 1 1 0.442 2.133 1.064
All events 298 0 1 1 1.570 2 5 1.136 2.618 0.371
Table. Number of service outcomes experienced
Number experienced Participants
0 58
1 92
2 82
3 55
4 8
5 3
Table. Proportion of participants having experienced number of service outcomes
Number experienced Proportion of participants
0 0.195
1 0.309
2 0.275
3 0.185
4 0.027
5 0.010
Table. Frequency of experience with service outcomes
House flooding Road flooding Cost change Waterbody pollution Bathing restrictions
0 260 100 196 247 219
1 38 198 102 51 79
Table. Experience with service outcomes (in %)
House flooding Road flooding Cost change Waterbody pollution Bathing restrictions
0 0.872 0.336 0.658 0.829 0.735
1 0.128 0.664 0.342 0.171 0.265

2.3.4 Response latency / decision time

(1) Time to complete survey overall

Min. 1st Qu. Median Mean 3rd Qu. Max. 9.116 24.800 30.823 33.477 40.200 88.582

(2) Response time per choice and by 1st series and 2nd series, by treatment condition and outcome domain

Table. Response latency (seconds per choice trial)
Set n min Q.25 median mean Q.75 max sd
By experimental condition
Both 298 1.456 7.461 10.483 11.169 13.869 44.672 5.722
Numeric 151 2.715 9.456 11.994 12.786 15.576 31.477 4.963
Icons 147 1.456 5.932 8.377 9.508 11.961 44.672 5.985
By elicitation series
1st series 298 1.323 8.949 12.345 13.788 17.225 80.703 8.087
2nd series 298 0.747 5.292 8.033 8.551 10.945 28.895 4.547
By outcome domain
Common domain (cost) 298 1.209 7.076 10.251 11.226 13.712 81.497 7.253
Priority domain 298 1.382 7.182 10.444 11.112 14.582 45.409 5.544
  • House flooding
152 1.382 7.972 11.236 12.138 15.453 45.409 5.992
  • Waterbody pollution
95 1.608 6.468 9.541 10.491 14.159 23.137 5.124
  • Road flooding
28 2.263 5.536 8.127 8.594 10.507 17.587 3.855
  • Bathing unavailable
23 2.585 6.244 9.073 9.964 12.758 19.548 4.439

(3) Time per gamble question

Table. Response latency on gambles – Common domain (cost)
Gamble min Q.25 median mean Q.75 max sd
1st trial series
G1 1.032 6.922 10.658 13.357 15.695 83.166 10.745
G2 0.824 5.237 8.370 11.008 13.310 102.553 9.592
G5 0.425 6.869 10.653 14.476 16.138 141.747 15.235
G6 0.842 6.474 9.239 12.489 14.502 139.664 12.516
G9 0.798 6.154 9.105 12.752 14.620 140.606 12.804
G10 0.838 5.862 9.411 14.008 14.774 241.098 21.870
G11 0.977 7.408 11.409 14.883 19.286 83.598 11.176
G12 0.753 6.464 10.044 12.614 15.346 128.204 11.491
G13 0.660 6.212 10.016 13.485 15.311 109.161 13.468
G14 0.856 6.774 10.610 15.216 19.025 137.704 15.091
G15 0.605 6.256 10.777 15.567 16.806 443.175 31.104
G17 0.886 6.033 9.740 12.758 15.605 176.361 13.567
G20 0.600 6.873 10.252 22.947 16.298 2916.051 168.542
G21 0.604 7.114 10.414 13.020 15.753 101.791 10.694
G22 0.854 7.045 9.914 13.123 14.223 159.101 15.139
G23 0.698 6.701 10.197 15.148 15.618 732.210 43.084
G24 0.821 5.902 8.310 10.904 12.680 87.655 9.293
G25 0.620 6.319 10.071 12.876 16.631 85.563 10.522
G26 1.124 7.527 11.088 13.801 16.061 103.864 12.186
G27 0.946 5.666 8.735 12.195 14.533 92.790 11.023
2nd trial series
G1 0.323 4.068 6.621 9.935 10.175 548.970 32.060
G2 0.337 3.768 6.442 7.831 9.605 133.224 9.026
G5 0.086 4.250 7.241 9.491 11.916 156.500 11.440
G6 0.007 4.065 6.672 8.794 10.490 92.067 8.760
G9 0.403 4.118 6.269 7.860 9.936 71.703 6.550
G10 0.313 3.971 6.249 7.986 10.204 68.087 7.275
G11 0.476 4.193 6.762 8.345 11.206 37.427 5.965
G12 0.334 4.031 6.708 9.020 11.005 233.255 14.568
G13 0.164 3.807 6.888 8.539 10.665 53.500 7.414
G14 0.388 4.528 7.020 8.863 10.927 153.956 10.244
G15 0.445 4.562 7.012 8.457 10.637 39.111 5.786
G17 0.405 4.307 6.871 8.076 10.450 40.721 5.676
G20 0.435 4.253 7.112 9.291 10.230 168.281 12.899
G21 0.636 4.151 6.294 8.047 9.719 64.012 7.060
G22 0.159 4.580 7.302 10.626 10.737 601.318 35.084
G23 0.652 4.022 6.184 9.014 9.791 290.168 18.031
G24 0.399 4.220 6.471 7.789 9.363 138.399 9.000
G25 0.315 4.160 6.766 8.205 10.189 44.584 6.299
G26 0.667 4.375 6.787 8.745 10.296 198.549 12.339
G27 0.289 3.996 6.141 7.513 9.380 92.308 6.899
Table. Response latency on gambles – Priority domain (other)
Gamble min Q.25 median mean Q.75 max sd
1st trial series
G1 0.660 7.678 11.206 14.478 17.903 88.339 12.173
G2 0.351 5.391 9.018 12.254 15.257 75.581 11.004
G5 0.607 6.600 10.032 12.443 14.447 80.811 10.528
G6 0.441 6.020 8.811 11.722 13.285 136.195 11.686
G9 0.256 7.353 11.961 15.143 17.906 224.850 17.372
G10 0.722 6.267 9.918 12.488 16.133 52.948 9.603
G11 0.240 7.774 11.334 16.579 20.005 164.931 17.452
G12 0.729 6.315 10.092 13.727 17.392 72.798 11.797
G13 0.391 6.218 9.899 13.747 15.844 216.185 16.565
G14 0.071 8.222 12.316 16.459 20.012 138.890 14.757
G15 0.469 6.884 10.736 13.352 16.107 96.333 11.450
G17 0.519 6.266 9.372 13.226 16.179 134.231 13.053
G20 0.423 7.341 11.905 14.750 17.087 192.093 14.279
G21 0.413 6.245 10.474 13.442 16.321 73.848 10.989
G22 0.468 6.650 10.503 12.833 14.729 125.896 11.753
G23 0.882 6.384 10.466 12.962 16.640 84.390 10.019
G24 0.276 5.497 8.531 10.967 13.988 106.745 9.527
G25 0.661 6.128 9.977 12.670 16.573 94.820 10.146
G26 0.130 8.229 12.674 16.529 20.867 103.994 13.057
G27 0.747 6.218 10.300 15.107 17.498 225.981 18.327
2nd trial series
G1 0.423 3.997 6.988 8.380 10.747 51.065 6.488
G2 0.122 3.781 6.418 7.723 9.389 105.185 7.806
G5 0.096 4.187 7.021 8.307 10.182 152.068 9.766
G6 0.546 3.950 6.603 7.969 9.501 62.156 7.086
G9 0.240 4.264 6.936 9.057 10.123 244.508 14.893
G10 0.413 3.850 6.716 8.274 10.181 49.980 7.003
G11 0.385 3.973 7.214 8.647 11.611 45.751 6.411
G12 0.358 3.532 6.896 8.774 10.530 145.785 11.831
G13 0.323 3.597 6.451 8.152 11.127 54.441 6.528
G14 0.560 4.512 7.362 9.423 11.280 110.241 9.089
G15 0.201 4.315 7.023 8.037 10.332 46.479 6.013
G17 0.416 4.077 6.749 9.124 10.149 293.728 17.856
G20 0.461 4.341 6.906 8.792 10.886 83.217 7.800
G21 0.509 3.924 7.170 8.465 11.297 39.720 6.561
G22 0.311 4.703 6.726 8.692 10.198 143.836 9.822
G23 0.091 3.812 6.554 8.906 11.023 269.537 16.648
G24 0.102 3.986 6.658 7.756 9.753 39.825 6.102
G25 0.221 3.738 6.890 8.380 11.178 69.079 7.178
G26 0.906 4.536 6.737 8.665 10.682 89.895 7.833
G27 0.415 3.782 6.527 8.086 10.421 56.136 6.330

2.4 Choice consistency

Consistency of choices regardless the implied (or induced) preference can also be measured using

  • Simple matching coefficient - as raw measure
  • Pearson’s Phi - same information, more common for psychos

On Pearson’s Phi

2.4.1 Simple matching coefficient (SMC)

Table. Consistency - simple matching coefficient
SMC n min Q.25 median mean Q.75 max sd kurtosis skewness
All gambles
both 298 0.300 0.575 0.650 0.651 0.725 0.950 0.121 2.870 -0.026
trees 151 0.300 0.575 0.625 0.636 0.700 0.900 0.115 3.206 -0.176
icons 147 0.350 0.575 0.675 0.666 0.750 0.950 0.125 2.501 0.034
By manipulation
M trials 298 0.125 0.500 0.625 0.636 0.750 1.000 0.191 2.762 -0.359
NM trials 298 0.250 0.562 0.656 0.655 0.750 0.969 0.127 2.984 -0.083
By outcome domain
Cost domain 298 0.150 0.550 0.700 0.666 0.750 1.000 0.154 3.203 -0.457
Priority domain 298 0.200 0.550 0.625 0.636 0.750 0.950 0.153 2.605 -0.054
– House flooding 152 0.250 0.538 0.600 0.627 0.750 0.950 0.149 2.479 -0.177
– Waterbody pollution 95 0.200 0.550 0.700 0.666 0.800 0.950 0.162 2.504 -0.083
– Road flooding 28 0.250 0.500 0.550 0.559 0.625 0.800 0.131 2.744 -0.224
– Bathing restictions 23 0.500 0.550 0.650 0.672 0.750 0.950 0.131 2.394 0.622
By manipulation and outcome domain
M cost domain 298 0.000 0.500 0.750 0.654 0.750 1.000 0.251 2.541 -0.375
M priority domain 298 0.200 0.550 0.625 0.636 0.750 0.950 0.153 2.605 -0.054
NM cost domain 298 0.125 0.562 0.688 0.669 0.812 1.000 0.164 3.186 -0.451
NM priority domain 298 0.188 0.562 0.625 0.641 0.750 1.000 0.162 2.587 -0.083

2.4.2 Pearson’s Phi

Table. Consistency - Pearson’s Phi coefficient
Phi n min Q.25 median mean Q.75 max sd kurtosis skewness
All gambles
all 298 -0.402 0.133 0.296 0.296 0.470 0.899 0.245 2.853 -0.025
trees 151 -0.402 0.100 0.257 0.265 0.404 0.814 0.233 3.150 -0.141
icons 147 -0.330 0.151 0.323 0.327 0.496 0.899 0.254 2.536 0.007
By manipulation
M trials 298 -0.775 -0.067 0.258 0.251 0.500 1.000 0.413 2.397 -0.352
NM trials 298 -0.500 0.148 0.284 0.305 0.473 0.939 0.256 2.969 -0.079
By outcome domain
cost domain 298 -0.698 0.123 0.328 0.329 0.533 1.000 0.314 3.135 -0.464
other domain 298 -0.612 0.051 0.246 0.267 0.492 0.905 0.310 2.572 -0.028
– floodh 152 -0.471 0.043 0.250 0.249 0.492 0.903 0.298 2.439 -0.129
– ekf 95 -0.612 0.092 0.341 0.331 0.589 0.905 0.330 2.482 -0.108
– floodr 28 -0.503 -0.025 0.086 0.099 0.248 0.601 0.267 2.776 -0.255
– bath 23 -0.042 0.110 0.218 0.320 0.481 0.899 0.274 2.402 0.648
By manipulation and outcome domain
NM cost domain 298 -0.750 0.129 0.358 0.337 0.618 1.000 0.332 3.111 -0.454
NM other domain 298 -0.630 0.073 0.258 0.279 0.520 1.000 0.328 2.554 -0.099

2.4.3 Rank stability between individuals based on rank from most risk seeking to least

## [1] 0.2871537
## [1] 0.2971246

3 Correlation and visualization of dependent and independent variables

3.1 Rank stability

(0) CBMD

(1) Domain rank

 bath ekf floodh floodr

0 7 7 12 5 2 4 15 36 6 4 4 14 34 1 6 5 19 17 6 8 1 9 18 4 10 2 13 10 3 12 0 7 11 2 14 0 6 6 0 16 0 3 4 0 18 0 0 2 1 20 0 1 0 0 22 0 1 2 0

(2) Experimental condition and stimuli type

Figure. Risk attitude by experimental condition

Figure. Risk attitude by experimental condition

Figure. Risk attitude change by experimental condition

Figure. Risk attitude change by experimental condition

Figure. Risk attitude stability by experimental condition

Figure. Risk attitude stability by experimental condition

Figure. Risk attitude difference on easy gambles by experimental condition

Figure. Risk attitude difference on easy gambles by experimental condition

Figure. Risk attitude difference on easy gambles by experimental condition

Figure. Risk attitude difference on easy gambles by experimental condition

Figure. Risk attitude difference on difficult gambles by experimental condition

Figure. Risk attitude difference on difficult gambles by experimental condition

Figure. Response time by experimental condition in first trial series

Figure. Response time by experimental condition in first trial series

Figure. Response time by experimental condition in 2nd trial series

Figure. Response time by experimental condition in 2nd trial series

(3) Cognitive style and ability, experience with WW service outcomes

Check for correlations and relationships between variables

Figure. Box plot risk.diff.abs ~ bnt,glt,bnt,crt, experience

Figure. Box plot risk.diff.abs ~ bnt,glt,bnt,crt, experience

(4) Response latency

(5) Also ran

3.2 CBMD

(0) Rank stability

(1) Domain rank

(2) Experimental condition and stimuli type

Figure. Number of choice concurrent detections by experimental condition

Figure. Number of choice concurrent detections by experimental condition

Figure. CBMD by gamble difficulty

Figure. CBMD by gamble difficulty

Figuure. CBMD on easy gambles

Figuure. CBMD on easy gambles

Figure. CBMD on difficult gambles

Figure. CBMD on difficult gambles

(3) Cognitive style and ability, experience with WW service outcomes

Figure. Box plot cbd.binary ~ cognitive abilities and experience

Figure. Box plot cbd.binary ~ cognitive abilities and experience

Figure. Box plot cbd.full ~ cognitive abilities and experience

Figure. Box plot cbd.full ~ cognitive abilities and experience

(4) Response latency

4 Coincidence of CBMD and preference stability across domains (RQ.1)

4.1 Rank stability from 1st to 2nd elicitation series

Hypothesis testing with cor.test(). using Kendall tau-b as more tractable than Spearman rho when ties are present, see Gilpin (1993) Educational and Psych. Measurement, 53(1):87-92

  • Overall rank stability is Kendall tau.b: 0.379, (z= 9.141, p-value <0.0001).

4.2 – H1.1 Risk preferences will be more stable in detected, manipulated trials than in manipulated but undetected trials.

(1) Is preference change on manipulated gambles lower for participants who corrected the manipulation?

Chi2 (one-sided, independent) test with H_0 that cbd and risk preference changes are independent of each other

  • H_0 not rejected
  • Chi2 X: 1.2199, p-value= 0.2694
  • DF: 1

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that mean change is lower for participants who corrected manipulation

  • H_0 not rejected
  • Wilcoxon W: 7971, z= -0.6198339, p-value= 0.2677
  • Difference in location: 0

table(risk.m.detected\(detected, abs(risk.m.detected\)difference), dnn = c(“detected”, “change”))

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that mean risk aversion score is lower for participants in the first trial series

  • H_0 not rejected
  • Wilcoxon W: 7971, z= -0.6198339, p-value= 0.2677
  • Difference in location: 0

table(risk.m.detected\(detected, abs(risk.m.detected\)difference), dnn = c(“detected”, “change”))

(2) Is there an association between preference change and concurrent detection?

Kendall tau-b rank correlation test (independent) test with H_0 that risk attidude score change decreases with detection:

  • H_0 not rejected
  • Kendalls tau-b: -1.1778, p-value= 0.2389, df=

(3) Did participants keep manipulated choice or not?

   0    1

0 1375 789 1 150 70

   0    1

0 6237 3299

4.3 – H1.2 Choice blindness detection and preference stability will be higher in domains that are perceived as more important.

** Main tests** #### (A) Test H0 that CBMD is not greater in domains that are more important (i.e. -rank) - across domains

  • Kendall tau-b: -0.009, (z= 0.234, p-value 0.5924).

(B) Test H0 that risk attitude difference is smaller (i.e. stability greater) in domains that are more important (i.e. -rank) - across domains

  • Kendall tau-b: -0.033, (z= -1.019, p-value 0.1542).

(C) Test H0 that SMC is not greater in domains that are more important (i.e. -rank) - across domains

  • Kendall tau-b: -0.011, (z= 0.364, p-value 0.6422).

(D) Quade test to test H0 that there is no difference between domains

  • CBMD
##          D  ND
## cost   107 248
## floodh  65 117
## floodr   7  22
## bath    12  19
## ekf     29  80
  • H_0 rejected
  • Quade F: 18, p-value= 0.0132, z = -2.219
  • Degrees of freedom: 1, 4

Post-hoc Pairwise Quade Test

Additional, exploratory tests #### (1) Testing for H0 that CBMD is the same for both domains, using Wilcoxon signed rank test

  • Wilcoxon signed rank test V: 899.5, (z= -0.557, p-value 0.2887).
  • Difference in location: 0

(2) Testing for H0 that RS is the same for both domains, using Wilcoxon signed rank test

  • Wilcoxon signed rank test V: 1.5822^{4}, (z= -0.457, p-value 0.3239).
  • Difference in location: 10^{-4}

(3) Testing for H0 that SMC is the same for both domains, using Wilcoxon signed rank test

  • Wilcoxon signed rank test V: 2.3098^{4}, (z= -3.248, p-value 6^{-4}).
  • Difference in location: 0.05

(4) Test H0 that CBMD is not greater in domains that are more important (i.e. -rank) - cost domain

  • Kendall tau-b: -0.013, (z= 0.257, p-value 0.6013).

(5) Test H0 that CBMD is not greater in domains that are more important (i.e. -rank) - other domain

  • Kendall tau-b: 0.104, (z= -1.88, p-value 0.0301).

(6) Test H0 that RS is not greater in domains that are more important (i.e. -rank) - cost domain

  • Kendall tau-b: 0.046, (z= -1.031, p-value 0.1512).

(7) Test H0 that RS is not greater in domains that are more important (i.e. -rank) - other domain

  • Kendall tau-b: 0.051, (z= -1.042, p-value 0.1486).

(8) Test H0 that SMC is not greater in domains that are more important (i.e. -rank) - cost domain

  • Kendall tau-b: 0.002, (z= -0.042, p-value 0.4832).

(9) Test H0 that SMC is not greater in domains that are more important (i.e. -rank) - other domain

  • Kendall tau-b: -0.06, (z= 1.235, p-value 0.8915).

4.4 Other analyses

(1) Gender effects on CRT, GLT, BNT, risk attitude, CBMD

CRT

GLT

BNT

Risk attitude

Risk attitude stability

CBMD

Gambles practice

(2) CRT effects on risk attitudes

CRT

5 To what extent do the elicitation stimuli influence choice blindness detection and preference stability? (RQ.2)

5.1 Hypothesis testing

H2.1 Choice blindness detection and preference stability will be higher in the treatment than baseline condition.

(1) Test H0~ that CBMD scores from icon condition are not higher than scores from tree condition

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that CBMD is higher in icon condition

  • H_0 not rejected
  • Wilcoxon W: 1.20285^{4}, p-value= 0.0503, z = -1.642
  • Difference in location: 1.6667297^{-5}

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that CBMD is higher in icon condition for those who corrected sth

  • H_0 not rejected
  • Wilcoxon W: 1664, p-value= 0.0655, z = -1.51
  • Difference in location: 3.5273089^{-5}

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that CBMD (binary) is higher in icon condition for those who corrected sth

  • H_0 not rejected
  • Wilcoxon W: 1.1993^{4}, p-value= 0.0549, z = -1.599
  • Difference in location: 5.1086323^{-5}

For exploration: Kolmogorov-Smirnov (one-sided, independent) test with H_0 that cdf distribution of CBMD is larger in icon condition

  • H_0 not rejected
  • Wilcoxon W: 0.0035, p-value= 0.9046, z = 1.308
  • Difference in location:

(2) Test H0~ that RS scores from icon condition are not lower than scores from tree condition

Wilcoxon (one-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that risk attitude stability is higher in icon condition

  • H_0 not rejected
  • Wilcoxon W: 1.0794^{4}, p-value= 0.3397, z = -0.413
  • Difference in location: 0

Wilcoxon (one-sided, independent) rank sum test with H_0 that risk attitude stability is higher in icon condition for those who corrected sth

  • H_0 not rejected
  • Wilcoxon W: 1545.5, p-value= 0.772, z = 0.746
  • Difference in location: 10^{-4}

(3) Test H0~ that risk attitude scores resulting from icon condition and tree condition are the same

Wilcoxon (two-sided, independent) rank sum test (= Mann-Whitney U test) with H_0 that risk attitude is same in icon condition

  • H_0 rejected
  • Wilcoxon W: 2274, p-value= 0, z = -11.819
  • Difference in location: -18

H2.2 CBMD and PS will be higher for distinct (easy) gambles than for similar (difficult) gambles, across both treatment conditions.

(1) Test H0~ that CBMD scores from easy gambles are not higher than scores from difficult gambles

Wilcoxon (one-sided, paired) rank sum test (= Mann-Whitney U test) with H_0 that concurrent detection is not higher for easy gambles

  • H_0 not rejected
  • Wilcoxon W: 868, p-value= 0.1888, z = -0.882
  • Difference in location: 0

Wilcoxon (one-sided, paired) rank sum test with H_0 that concurrent detection is not higher for easy gambles for those who corrected sth

  • H_0 not rejected
  • Wilcoxon W: 868, p-value= 0.1888, z = -0.882
  • Difference in location: 0

Quade test to understand whether there is a difference among gambles G5-G21

##     icon tree
## G5    30   20
## G9    29   25
## G10   33   22
## G21   37   24
  • H_0 rejected
  • Quade F: 15, p-value= 0.0305, z = -1.874
  • Degrees of freedom: 1, 3

Post-hoc Pairwise Quade Test

Wilcoxon (one-sided, independent rank sum test with H_0 that concurrent detection is not higher for easy gambles in icon condition

  • H_0 not rejected
  • Wilcoxon W: 1.1724^{4}, p-value= 0.122, z = -1.165
  • Difference in location: 0

Wilcoxon (one-sided, independent) rank sum test with H_0 that concurrent detection is not higher difficulteasy gambles in icon condition

  • H_0 rejected
  • Wilcoxon W: 1.1898^{4}, p-value= 0.0493, z = -1.652
  • Difference in location: 0

(2) Test H0~ that RS scores from easy gambles are more stable than scores from difficult gambles

Wilcoxon (one-sided, paired) rank sum test (= Mann-Whitney U test) with H_0 that risk attitude difference is not lower for easy gambles

  • H_0 rejected
  • Wilcoxon W: 6551, p-value= 0.0047, z = -2.598
  • Difference in location: -2.0030775^{-5}

Quade test to understand whether there is a difference in changes among gambles G5-G21

##     icon tree
## G5    73   80
## G9    91  111
## G10   82   93
## G21   85  104
  • H_0 rejected
  • Quade F: 15, p-value= 0.0305, z = -1.874
  • Degrees of freedom: 1, 3

Post-hoc Pairwise Quade Test

Plot descriptive stats side by side
Table. Concurrent detection and risk attitude changes by gamble pair
Concurrent detection
Risk attitude changes
G5 G10 G9 G21 G5 G10 G9 G21
icon 30 33 29 37 73 82 91 85
tree 20 22 25 24 80 93 111 104

H2.3 - Removed (outcome manipulations vs. probability manipulation)

6 Which individual characteristics may explain differences in choice blindness detection and preference stability? (RQ.3)

across domains and treatment conditions

6.1 Hypothesis testing

H3.1a - Choice blindness increases if the respondents…

(a.) preferences about the gamble are weakly held (close to indifferent) * I.e. how long did person take on this gamble compared to others? * Or use indifference statements (b.+c.) cognitive reasoning, risk numeracy, and graph literacy scores are low (d.) experience or familiarity with domain are low (e.) perceived importance of domain is low

(1) Calculate and plot cross-correlation matrix and test significance

Cross-correlation matrix of DVs and IVs
change cbd rpe_time icon crt bnt glt delta.ev.ref delta.p g5 g9 g10 g21 d.cost d.bath d.ekf d.floodh d.floodr experience rank age female
change 1.00 -0.03 -0.03 -0.03 0.00 -0.06 0.01 0.04 0.01 -0.04 -0.01 0.03 0.02 -0.03 -0.02 -0.02 0.06 0.01 0.02 -0.01 0.02 0.02
d.floodh 0.06 0.03 0.09 0.02 0.07 0.04 0.03 0.00 0.00 0.00 0.00 0.00 0.00 -0.59 -0.12 -0.25 1.00 -0.13 -0.21 0.08 0.10 -0.02
delta.ev.ref 0.04 0.02 0.00 0.00 0.00 0.00 0.00 1.00 0.45 -0.77 -0.26 0.26 0.77 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
g10 0.03 0.00 -0.04 0.00 0.00 0.00 0.00 0.26 -0.58 -0.33 -0.33 1.00 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
g21 0.02 0.02 0.02 0.00 0.00 0.00 0.00 0.77 0.58 -0.33 -0.33 -0.33 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
experience 0.02 -0.03 -0.05 0.00 -0.11 -0.06 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.14 0.03 -0.09 -0.21 0.23 1.00 0.05 0.02 0.04
age 0.02 -0.13 0.14 0.00 0.12 -0.11 -0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.10 -0.06 0.10 -0.02 0.02 0.16 1.00 0.02
female 0.02 -0.05 -0.03 -0.02 -0.22 -0.11 -0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.02 -0.02 -0.02 0.04 -0.04 0.02 1.00
glt 0.01 0.04 0.02 -0.01 0.28 0.13 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.06 -0.03 0.03 0.05 0.00 -0.08 -0.03 -0.05
delta.p 0.01 0.01 0.03 0.00 0.00 0.00 0.00 0.45 1.00 -0.58 0.58 -0.58 0.58 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
d.floodr 0.01 -0.02 -0.05 -0.01 -0.06 -0.05 0.05 0.00 0.00 0.00 0.00 0.00 0.00 -0.22 -0.04 -0.10 -0.13 1.00 0.23 0.01 -0.02 -0.02
crt 0.00 0.08 0.11 0.00 1.00 0.45 0.28 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.05 -0.02 0.07 -0.06 -0.11 0.09 0.12 -0.22
g9 -0.01 0.00 0.02 0.00 0.00 0.00 0.00 -0.26 0.58 -0.33 1.00 -0.33 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
rank -0.01 0.03 0.08 0.02 0.09 0.17 -0.08 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.09 -0.05 0.08 0.01 0.05 1.00 0.16 -0.04
d.bath -0.02 0.03 -0.03 -0.01 -0.05 0.01 -0.06 0.00 0.00 0.00 0.00 0.00 0.00 -0.20 1.00 -0.09 -0.12 -0.04 0.03 -0.09 -0.10 0.03
d.ekf -0.02 -0.02 -0.04 -0.02 -0.02 -0.02 -0.03 0.00 0.00 0.00 0.00 0.00 0.00 -0.44 -0.09 1.00 -0.25 -0.10 -0.09 -0.05 -0.06 0.02
cbd -0.03 1.00 0.10 0.06 0.08 0.16 0.04 0.02 0.01 -0.02 0.00 0.00 0.02 -0.01 0.03 -0.02 0.03 -0.02 -0.03 0.03 -0.13 -0.05
rpe_time -0.03 0.10 1.00 -0.30 0.11 0.06 0.02 0.00 0.03 0.00 0.02 -0.04 0.02 -0.02 -0.03 -0.04 0.09 -0.05 -0.05 0.08 0.14 -0.03
icon -0.03 0.06 -0.30 1.00 0.00 0.06 -0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.01 -0.02 0.02 -0.01 0.00 0.02 0.00 -0.02
d.cost -0.03 -0.01 -0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 -0.20 -0.44 -0.59 -0.22 0.14 0.00 0.00 0.00
g5 -0.04 -0.02 0.00 0.00 0.00 0.00 0.00 -0.77 -0.58 1.00 -0.33 -0.33 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bnt -0.06 0.16 0.06 0.06 0.45 1.00 0.13 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 -0.02 0.04 -0.05 -0.06 0.17 -0.11 -0.11
  • Plot cross-correlation matrix, diplaying significant correlations only
    Cross-correlations of DVs and IVs. Only significant correlations (p<.25) are shown

    Cross-correlations of DVs and IVs. Only significant correlations (p<.25) are shown

(2) Define which variables to include

Using mixed effect logistic regression with intercept and slope by individual, applying selection rules in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2633005/

Starting with full model

  • Note that interaction rank:bnt was ignored as no reasonable explanation apparent and also hardly any effect on model predictive capacity, i.e. assuming it as spurious relationship that can be discarded.

  • candidates for removal are (p <.1): glt, crt, bnt, experience, rank, female

  • test for confounders (if removal leads to change in another predictor > 15-20%)

  • remove (indiff) female, rank, glt

  • interaction effect of d.bath:experience only adds inf error, but no change in odds, kept interactions still

  • Can CRT be deleted without issues?

  • adding back variables not considered earlier: delta.ev.ref, delta.p, g5, g9, g10, g21, d.cost, d.floodr and keep if p<.1

(3) See whether reduced model really improved upon extensive model

Which model improves upon earlier or not?

  • Small difference between models only. model 2 best

(4) Residuals diagnosis

Residuals analysis using DHARMa package
Figure. Residual diagnostics for model cbd.i2

Figure. Residual diagnostics for model cbd.i2

$uniformity

Asymptotic one-sample Kolmogorov-Smirnov test

data: simulationOutput$scaledResiduals D = 0.017739, p-value = 0.4412 alternative hypothesis: two-sided

$dispersion

DHARMa nonparametric dispersion test via sd of residuals fitted vs.
simulated

data: simulationOutput dispersion = 0.98767, p-value = 0.992 alternative hypothesis: two.sided

$outliers

DHARMa outlier test based on exact binomial test with approximate
expectations

data: simulationOutput outliers at both margin(s) = 12, observations = 2384, p-value = 0.1321 alternative hypothesis: true probability of success is not equal to 0.007968127 95 percent confidence interval: 0.002603536 0.008776087 sample estimates: frequency of outliers (expected: 0.00796812749003984 ) 0.005033557

$uniformity

Asymptotic one-sample Kolmogorov-Smirnov test

data: simulationOutput$scaledResiduals D = 0.017739, p-value = 0.4412 alternative hypothesis: two-sided

$dispersion

DHARMa nonparametric dispersion test via sd of residuals fitted vs.
simulated

data: simulationOutput dispersion = 0.98767, p-value = 0.992 alternative hypothesis: two.sided

$outliers

DHARMa outlier test based on exact binomial test with approximate
expectations

data: simulationOutput outliers at both margin(s) = 12, observations = 2384, p-value = 0.1321 alternative hypothesis: true probability of success is not equal to 0.007968127 95 percent confidence interval: 0.002603536 0.008776087 sample estimates: frequency of outliers (expected: 0.00796812749003984 ) 0.005033557

(5) Plot regression tables, predictions etc

  • Plot regression tables
    Effect of explanatory variables on (concurrent) choice blindness detection
      Mixed effects logit model (all vars) Mixed effects logit model (final)
    Predictors Odds Ratios std. Error CI p Odds Ratios std. Error CI p
    Intercept 0.00 0.00 0.00 – 0.09 0.001 0.00 0.00 0.00 – 0.03 <0.001
    Choice latency [log] 1.80 0.33 1.26 – 2.57 0.001 1.80 0.33 1.26 – 2.58 0.001
    Icon condition 2.70 1.55 0.88 – 8.33 0.083 2.65 1.52 0.86 – 8.16 0.089
    CRT score 1.12 0.87 0.24 – 5.13 0.884
    GLT score 0.81 0.63 0.17 – 3.75 0.786
    BNT score 2.02 1.00 0.77 – 5.33 0.153 1.86 0.49 1.11 – 3.13 0.018
    D: Bathing restrictions 1.00 0.63 0.29 – 3.41 0.999 0.98 0.61 0.29 – 3.34 0.975
    D: Waterbody pollution 0.26 0.14 0.09 – 0.73 0.011 0.26 0.14 0.09 – 0.73 0.010
    D: House flooding 1.97 0.59 1.10 – 3.54 0.023 1.99 0.60 1.11 – 3.58 0.022
    Experience 0.73 0.28 0.35 – 1.55 0.415 0.73 0.28 0.35 – 1.54 0.406
    Domain rank 1.16 0.28 0.72 – 1.87 0.548
    Age 0.92 0.02 0.88 – 0.97 0.001 0.93 0.02 0.89 – 0.97 0.001
    Female 0.99 0.58 0.31 – 3.11 0.985
    CRT:GLT 1.12 0.50 0.47 – 2.68 0.792
    CRT:BNT 0.89 0.22 0.55 – 1.43 0.627
    D:E Waterbody pollution 4.92 4.66 0.77 – 31.51 0.092 5.03 4.78 0.78 – 32.33 0.089
    D:E House flooding 1.68 1.43 0.32 – 8.92 0.542 1.65 1.39 0.31 – 8.64 0.556
    Random Effects
    σ2 3.29 3.29
    τ00 15.74 participant.id 16.20 participant.id
    ICC 0.83 0.83
    N 298 participant.id 298 participant.id
    Observations 2384 2384
    Marginal R2 / Conditional R2 0.111 / 0.846 0.103 / 0.849
    Deviance 569.524 568.860
    AIC 980.536 969.532
    AICc 980.825 969.664
    log-Likelihood -472.268 -472.766
    • Plot fixed effects and predictions of marginal effects 
    ## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus

    ## NOTE: Results may be misleading due to involvement in interactions

    ## NOTE: Results may be misleading due to involvement in interactions

    ## Data were 'prettified'. Consider using `terms="age [all]"` to get smooth
##   plots.

(6) Additional tests for effect of experience H3.1d)

COST: McNemar’s (one-sided, paired) ^2 test with H_0 that concurrent detection is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Odds ratio: 2.4054, p-value= 0, z = -4.606

EKF: McNemar’s (one-sided, paired) ^2 test with H_0 that concurrent detection is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Odds ratio: 5, p-value= 0, z = -4.954

FLOOD-H: McNemar’s (one-sided, paired) ^2 test with H_0 that concurrent detection is not higher for individuals who have experienced outcome

  • H_0 not rejected
  • Odds ratio: 1.0968, p-value= 0.4022, z = -0.248

FLOOD-R: McNemar’s (one-sided, paired) ^2 test with H_0 that concurrent detection is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Odds ratio: 97, p-value= 0, z = -15.641

BATH: McNemar’s (one-sided, paired) ^2 test with H_0 that concurrent detection is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Odds ratio: 19.75, p-value= 0, z = -8.937

H3.1b - Preference instability increases if the respondents…

(a.) preferences about the gamble are weakly held (close to indifferent) * I.e. how long did person take on this gamble compared to others? * Or use indifference statements (b.+c.) cognitive reasoning, risk numeracy, and graph literacy scores are low (d.) experience or familiarity with domain are low (e.) perceived importance of domain is low #### (a.) preferences about the gamble are weakly held (close to indifferent) + I.e. how long did person take on this gamble compared to others? + Or use indifference statements

(1) Cross-correlation matrix for risk preference stability

  • Choice (preference) stability –> see cross-correl matrix for CBMD
  • Change direction (-1 = risk seeking, 1 = risk averse) –> would need ordinal regression analysis, do later if time
    Cross-correlation matrix of risk attitude change
    change cbd rpe_time icon crt bnt glt delta.ev.ref delta.p g5 g9 g10 g21 d.cost d.bath d.ekf d.floodh d.floodr experience rank age female
    change 1.00 -0.03 -0.03 -0.03 0.00 -0.06 0.01 0.04 0.01 -0.04 -0.01 0.03 0.02 -0.03 -0.02 -0.02 0.06 0.01 0.02 -0.01 0.02 0.02
    d.floodh 0.06 0.03 0.09 0.02 0.07 0.04 0.03 0.00 0.00 0.00 0.00 0.00 0.00 -0.59 -0.12 -0.25 1.00 -0.13 -0.21 0.08 0.10 -0.02
    delta.ev.ref 0.04 0.02 0.00 0.00 0.00 0.00 0.00 1.00 0.45 -0.77 -0.26 0.26 0.77 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    g10 0.03 0.00 -0.04 0.00 0.00 0.00 0.00 0.26 -0.58 -0.33 -0.33 1.00 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    g21 0.02 0.02 0.02 0.00 0.00 0.00 0.00 0.77 0.58 -0.33 -0.33 -0.33 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    experience 0.02 -0.03 -0.05 0.00 -0.11 -0.06 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.14 0.03 -0.09 -0.21 0.23 1.00 0.05 0.02 0.04
    age 0.02 -0.13 0.14 0.00 0.12 -0.11 -0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.10 -0.06 0.10 -0.02 0.02 0.16 1.00 0.02
    female 0.02 -0.05 -0.03 -0.02 -0.22 -0.11 -0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.02 -0.02 -0.02 0.04 -0.04 0.02 1.00
    glt 0.01 0.04 0.02 -0.01 0.28 0.13 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.06 -0.03 0.03 0.05 0.00 -0.08 -0.03 -0.05
    delta.p 0.01 0.01 0.03 0.00 0.00 0.00 0.00 0.45 1.00 -0.58 0.58 -0.58 0.58 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    d.floodr 0.01 -0.02 -0.05 -0.01 -0.06 -0.05 0.05 0.00 0.00 0.00 0.00 0.00 0.00 -0.22 -0.04 -0.10 -0.13 1.00 0.23 0.01 -0.02 -0.02
    crt 0.00 0.08 0.11 0.00 1.00 0.45 0.28 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.05 -0.02 0.07 -0.06 -0.11 0.09 0.12 -0.22
    g9 -0.01 0.00 0.02 0.00 0.00 0.00 0.00 -0.26 0.58 -0.33 1.00 -0.33 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    rank -0.01 0.03 0.08 0.02 0.09 0.17 -0.08 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.09 -0.05 0.08 0.01 0.05 1.00 0.16 -0.04
    d.bath -0.02 0.03 -0.03 -0.01 -0.05 0.01 -0.06 0.00 0.00 0.00 0.00 0.00 0.00 -0.20 1.00 -0.09 -0.12 -0.04 0.03 -0.09 -0.10 0.03
    d.ekf -0.02 -0.02 -0.04 -0.02 -0.02 -0.02 -0.03 0.00 0.00 0.00 0.00 0.00 0.00 -0.44 -0.09 1.00 -0.25 -0.10 -0.09 -0.05 -0.06 0.02
    cbd -0.03 1.00 0.10 0.06 0.08 0.16 0.04 0.02 0.01 -0.02 0.00 0.00 0.02 -0.01 0.03 -0.02 0.03 -0.02 -0.03 0.03 -0.13 -0.05
    rpe_time -0.03 0.10 1.00 -0.30 0.11 0.06 0.02 0.00 0.03 0.00 0.02 -0.04 0.02 -0.02 -0.03 -0.04 0.09 -0.05 -0.05 0.08 0.14 -0.03
    icon -0.03 0.06 -0.30 1.00 0.00 0.06 -0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.01 -0.02 0.02 -0.01 0.00 0.02 0.00 -0.02
    d.cost -0.03 -0.01 -0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 -0.20 -0.44 -0.59 -0.22 0.14 0.00 0.00 0.00
    g5 -0.04 -0.02 0.00 0.00 0.00 0.00 0.00 -0.77 -0.58 1.00 -0.33 -0.33 -0.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    bnt -0.06 0.16 0.06 0.06 0.45 1.00 0.13 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 -0.02 0.04 -0.05 -0.06 0.17 -0.11 -0.11
  • Plot cross-correlation matrix, diplaying significant correlations only
## Warning in corrplot(corr, type = "upper", method = upper, diag = TRUE, tl.pos =
## tl.pos, : p.mat and corr may be not paired, their rownames and colnames are not
## totally same!
## Warning in corrplot(corr, add = TRUE, type = "lower", method = lower, diag =
## (diag == : p.mat and corr may be not paired, their rownames and colnames are
## not totally same!
Cross-correlations of risk attitude change with explantory variables. Only significant correlations (p<.25) are shown

Cross-correlations of risk attitude change with explantory variables. Only significant correlations (p<.25) are shown

(2) Define which variables to include

Using mixed effect logistic regression with intercept and slope by individual, applying selection rules in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2633005/

Starting with full model

  • candidates for removal are (p <.1): cbd, log(rpe_time), icon, delta.ev.ref, d.cost, d.ekf, experience

  • test for confounders (if removal leads to change in another predictor > 15-20%)

  • remove cbd, delta.ev.ref

  • Relevant if adding G5 and G10 or delta.ev.ref?

  • adding unconsidered variables and keep if p<.15: female, age, delta.p, g9, g21, d.bath, d.floodr, crt, glt, expfloodr, expbathing, rank

  • any irrelevant interactions e.g. bnt:cbd, icon:log(rpe_time)

tab_model(chg.logit, chg.logit.2, chg.logit.3)

(3) See whether reduced model really improved upon extensive model

Which model improves upon earlier or not?

  • Small difference between models only. LRT not significant, AIC smaller for smaller model

(4) Residuals diagnosis

Residuals analysis using DHARMa package
Figure. Residual diagnostics

Figure. Residual diagnostics

$uniformity

Asymptotic one-sample Kolmogorov-Smirnov test

data: simulationOutput$scaledResiduals D = 0.015744, p-value = 0.5957 alternative hypothesis: two-sided

$dispersion

DHARMa nonparametric dispersion test via sd of residuals fitted vs.
simulated

data: simulationOutput dispersion = 0.9999, p-value = 0.968 alternative hypothesis: two.sided

$outliers

DHARMa outlier test based on exact binomial test with approximate
expectations

data: simulationOutput outliers at both margin(s) = 18, observations = 2384, p-value = 0.9085 alternative hypothesis: true probability of success is not equal to 0.007968127 95 percent confidence interval: 0.004480767 0.011906630 sample estimates: frequency of outliers (expected: 0.00796812749003984 ) 0.007550336

$uniformity

Asymptotic one-sample Kolmogorov-Smirnov test

data: simulationOutput$scaledResiduals D = 0.015744, p-value = 0.5957 alternative hypothesis: two-sided

$dispersion

DHARMa nonparametric dispersion test via sd of residuals fitted vs.
simulated

data: simulationOutput dispersion = 0.9999, p-value = 0.968 alternative hypothesis: two.sided

$outliers

DHARMa outlier test based on exact binomial test with approximate
expectations

data: simulationOutput outliers at both margin(s) = 18, observations = 2384, p-value = 0.9085 alternative hypothesis: true probability of success is not equal to 0.007968127 95 percent confidence interval: 0.004480767 0.011906630 sample estimates: frequency of outliers (expected: 0.00796812749003984 ) 0.007550336

  • some underdispersion is present, standard ways to solve this would be using hurdle models or similar

(5) Plot regression tables, predictions etc

  • Plot regression tables
    Effect of explanatory variables on preference stability
      Mixed effects logit model (all vars) Mixed effects logit model (final)
    Predictors Odds Ratios std. Error CI p Odds Ratios std. Error CI p
    Intercept 0.83 0.23 0.48 – 1.43 0.499 0.86 0.24 0.49 – 1.48 0.576
    Concurrent detection 0.88 0.26 0.50 – 1.56 0.665
    Choice latency [log] 0.90 0.06 0.80 – 1.02 0.093 0.90 0.06 0.80 – 1.01 0.082
    Icon condition 0.86 0.09 0.71 – 1.04 0.129 0.85 0.09 0.70 – 1.04 0.116
    BNT score 0.91 0.04 0.83 – 1.00 0.045 0.91 0.04 0.83 – 0.99 0.030
    EV spread 2.31 1.21 0.82 – 6.46 0.111
    D: Cost 0.93 0.23 0.57 – 1.51 0.769 0.94 0.23 0.57 – 1.53 0.793
    D: Waterbody pollution 0.96 0.26 0.56 – 1.62 0.868 0.97 0.26 0.57 – 1.64 0.906
    D: House flooding 1.23 0.31 0.75 – 2.03 0.418 1.24 0.32 0.75 – 2.04 0.402
    Experience 0.91 0.29 0.49 – 1.69 0.771 0.92 0.29 0.50 – 1.71 0.801
    D:E Cost 1.28 0.44 0.65 – 2.51 0.469 1.27 0.44 0.65 – 2.49 0.487
    D:E Waterbody pollution 0.92 0.40 0.39 – 2.16 0.849 0.90 0.39 0.38 – 2.10 0.802
    D:E House flooding 1.86 0.77 0.82 – 4.18 0.135 1.83 0.76 0.81 – 4.13 0.143
    CBMD:BNT 1.01 0.15 0.76 – 1.36 0.923
    Gamble pair G5 0.81 0.08 0.66 – 0.99 0.037
    Random Effects
    σ2 3.29 3.29
    τ00 0.10 participant.id 0.11 participant.id
    ICC 0.03 0.03
    N 298 participant.id 298 participant.id
    Observations 2384 2384
    Marginal R2 / Conditional R2 0.016 / 0.046 0.017 / 0.047
    Deviance 2984.196 2980.560
    AIC 3111.701 3106.174
    AICc 3111.904 3106.327
    log-Likelihood -1540.851 -1540.087
    • Plot fixed effects and predictions of marginal effects 
    ## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus
## Warning in sprintf("%f", abs(x)%%1): probable complete loss of accuracy in
## modulus

(6) Plot regression tables for CBMD and preference change in one

Mixed effects logistic regression for concurrent detections and preference stability
  Concurrent detections (all variables) Concurrent detections (final) Preference change (all variables) Preference change (final)
Predictors Odds Ratios SE CI p Odds Ratios SE CI p Odds Ratios SE CI p Odds Ratios SE CI p
Intercept 0.00 0.00 0.00 – 0.09 0.001 0.00 0.00 0.00 – 0.03 <0.001 0.83 0.23 0.48 – 1.43 0.499 0.86 0.24 0.49 – 1.48 0.576
Choice latency [log] 1.80 0.33 1.26 – 2.57 0.001 1.80 0.33 1.26 – 2.58 0.001 0.90 0.06 0.80 – 1.02 0.093 0.90 0.06 0.80 – 1.01 0.082
Icon condition 2.70 1.55 0.88 – 8.33 0.083 2.65 1.52 0.86 – 8.16 0.089 0.86 0.09 0.71 – 1.04 0.129 0.85 0.09 0.70 – 1.04 0.116
Concurrent detection 0.88 0.26 0.50 – 1.56 0.665
EV spread 2.31 1.21 0.82 – 6.46 0.111
Gamble pair G5 0.81 0.08 0.66 – 0.99 0.037
CRT score 1.12 0.87 0.24 – 5.13 0.884
GLT score 0.81 0.63 0.17 – 3.75 0.786
BNT score 2.02 1.00 0.77 – 5.33 0.153 1.86 0.49 1.11 – 3.13 0.018 0.91 0.04 0.83 – 1.00 0.045 0.91 0.04 0.83 – 0.99 0.030
D: Bathing restrictions 1.00 0.63 0.29 – 3.41 0.999 0.98 0.61 0.29 – 3.34 0.975
D: Cost 0.93 0.23 0.57 – 1.51 0.769 0.94 0.23 0.57 – 1.53 0.793
D: House flooding 1.97 0.59 1.10 – 3.54 0.023 1.99 0.60 1.11 – 3.58 0.022 1.23 0.31 0.75 – 2.03 0.418 1.24 0.32 0.75 – 2.04 0.402
D: Waterbody pollution 0.26 0.14 0.09 – 0.73 0.011 0.26 0.14 0.09 – 0.73 0.010 0.96 0.26 0.56 – 1.62 0.868 0.97 0.26 0.57 – 1.64 0.906
Domain rank 1.16 0.28 0.72 – 1.87 0.548
Experience 0.73 0.28 0.35 – 1.55 0.415 0.73 0.28 0.35 – 1.54 0.406 0.91 0.29 0.49 – 1.69 0.771 0.92 0.29 0.50 – 1.71 0.801
Age 0.92 0.02 0.88 – 0.97 0.001 0.93 0.02 0.89 – 0.97 0.001
Female 0.99 0.58 0.31 – 3.11 0.985
CBMD:BNT 1.01 0.15 0.76 – 1.36 0.923
CRT:GLT 1.12 0.50 0.47 – 2.68 0.792
CRT:BNT 0.89 0.22 0.55 – 1.43 0.627
D:E Waterbody pollution 4.92 4.66 0.77 – 31.51 0.092 5.03 4.78 0.78 – 32.33 0.089 0.92 0.40 0.39 – 2.16 0.849 0.90 0.39 0.38 – 2.10 0.802
D:E House flooding 1.68 1.43 0.32 – 8.92 0.542 1.65 1.39 0.31 – 8.64 0.556 1.86 0.77 0.82 – 4.18 0.135 1.83 0.76 0.81 – 4.13 0.143
D:E Cost 1.28 0.44 0.65 – 2.51 0.469 1.27 0.44 0.65 – 2.49 0.487
Random Effects
σ2 3.29 3.29 3.29 3.29
τ00 15.74 participant.id 16.20 participant.id 0.10 participant.id 0.11 participant.id
ICC 0.83 0.83 0.03 0.03
N 298 participant.id 298 participant.id 298 participant.id 298 participant.id
Observations 2384 2384 2384 2384
Marginal R2 / Conditional R2 0.111 / 0.846 0.103 / 0.849 0.016 / 0.046 0.017 / 0.047
Deviance 569.524 568.860 2984.196 2980.560
AIC 980.536 969.532 3111.701 3106.174
AICc 980.825 969.664 3111.904 3106.327
log-Likelihood -472.268 -472.766 -1540.851 -1540.087

(7) Additional tests for effect of experience H3.1d)

COST: Wilcoxon (one-sided, paired) signed rank test with H_0 that risk attitude change is not higher for individuals who have experienced outcome

  • H_0 rejected

  • Estimate (pseudo-median): 4, p-value= 0, z = -13.958 EKF: Wilcoxon (one-sided, paired) signed rank test with H_0 that risk attitude change is not higher for individuals who have experienced outcome

  • H_0 rejected

  • Estimate (pseudo-median): 4.0001, p-value= 0, z = -13.737

FLOOD-H: Wilcoxon (one-sided, paired) signed rank test with H_0 that risk attitude change is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Estimate (pseudo-median): 4, p-value= 0, z = -13.685

FLOOD-R: Wilcoxon (one-sided, paired) signed rank test with H_0 that risk attitude change is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Estimate (pseudo-median): 3.0001, p-value= 0, z = -13.287

FLOOD-R: Wilcoxon (one-sided, paired) signed rank test with H_0 that risk attitude change is not higher for individuals who have experienced outcome

  • H_0 rejected
  • Estimate (pseudo-median): 4.0001, p-value= 0, z = -13.709