R/sig_fit_bootstrap_batch.R
sig_fit_bootstrap_batch.RdRead sig_fit_bootstrap for more option setting.
a numeric matrix V with row representing components and
columns representing samples, typically you can get nmf_matrix from sig_tally() and
transpose it by t().
a subset of c("NNLS", "QP", "SA").
the number of bootstrap replicates.
minimal exposure in a sample, default is 1. Any patient has total exposure less than this value will be filtered out.
a vector of relative exposure threshold for calculating p values.
if TRUE, use parallel computation based on furrr package.
It can also be an integer for specifying cores.
random seed to reproduce the result.
a job ID, default is NULL, can be a string. When not NULL, all bootstrapped results
will be saved to local machine location defined by result_dir. This is very useful for running
more than 10 times for more than 100 samples.
see above, default is temp directory defined by R.
other common parameters passing to sig_fit_bootstrap, including
sig, sig_index, sig_db, db_type, mode, auto_reduce etc.
a list of data.table.
# For mutational signatures ----------------
# SBS is used for illustration, similar
# operations can be applied to DBS, INDEL, CN, RS, etc.
# Load simulated data
data("simulated_catalogs")
data = simulated_catalogs$set1
data[1:5, 1:5]
#> Sample_1 Sample_2 Sample_3 Sample_4 Sample_5
#> A[C>A]A 911 195 95 131 33
#> A[C>A]C 761 175 51 71 10
#> A[C>A]G 88 19 12 14 2
#> A[C>A]T 744 174 55 77 14
#> C[C>A]A 883 225 142 170 55
# Fitting with COSMIC reference signatures
# \donttest{
# Generally set n = 100
rv = sig_fit_bootstrap_batch(data,
sig_index = c(1, 5, 9, 2, 13),
sig_db = "SBS", n = 10)
#> ℹ [2024-08-04 14:44:55.044592]: Batch Bootstrap Signature Exposure Analysis Started.
#> ℹ [2024-08-04 14:44:55.046463]: Samples to be filtered out:
#> ℹ [2024-08-04 14:44:55.048015]: Finding optimal exposures (&errors) for different methods.
#> ℹ [2024-08-04 14:44:55.049417]: Calling method `QP`.
#> ℹ [2024-08-04 14:44:55.052672]: Started.
#> ✔ [2024-08-04 14:44:55.054073]: Signature index detected.
#> ℹ [2024-08-04 14:44:55.055418]: Checking signature database in package.
#> ℹ [2024-08-04 14:44:55.058596]: Checking signature index.
#> ℹ [2024-08-04 14:44:55.060048]: Valid index for db 'SBS':
#> 1 2 3 4 5 6 7a 7b 7c 7d 8 9 10a 10b 11 12 13 14 15 16 17a 17b 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 84 85 86 87 88 89 90
#> ✔ [2024-08-04 14:44:55.061507]: Database and index checked.
#> ✔ [2024-08-04 14:44:55.062983]: Signature normalized.
#> ℹ [2024-08-04 14:44:55.064329]: Checking row number for catalog matrix and signature matrix.
#> ✔ [2024-08-04 14:44:55.065761]: Checked.
#> ℹ [2024-08-04 14:44:55.067096]: Checking rownames for catalog matrix and signature matrix.
#> ℹ [2024-08-04 14:44:55.068444]: Matrix V and W don't have same orders. Try reordering...
#> ✔ [2024-08-04 14:44:55.069968]: Checked.
#> ✔ [2024-08-04 14:44:55.071327]: Method 'QP' detected.
#> ✔ [2024-08-04 14:44:55.072694]: Corresponding function generated.
#> ℹ [2024-08-04 14:44:55.074085]: Calling function.
#> ℹ [2024-08-04 14:44:55.075759]: Fitting sample: Sample_1
#> ℹ [2024-08-04 14:44:55.077262]: Fitting sample: Sample_2
#> ℹ [2024-08-04 14:44:55.078694]: Fitting sample: Sample_3
#> ℹ [2024-08-04 14:44:55.080123]: Fitting sample: Sample_4
#> ℹ [2024-08-04 14:44:55.081548]: Fitting sample: Sample_5
#> ℹ [2024-08-04 14:44:55.082973]: Fitting sample: Sample_6
#> ℹ [2024-08-04 14:44:55.084372]: Fitting sample: Sample_7
#> ℹ [2024-08-04 14:44:55.085851]: Fitting sample: Sample_8
#> ℹ [2024-08-04 14:44:55.087266]: Fitting sample: Sample_9
#> ℹ [2024-08-04 14:44:55.088704]: Fitting sample: Sample_10
#> ℹ [2024-08-04 14:44:55.090147]: Fitting sample: Sample_11
#> ℹ [2024-08-04 14:44:55.091571]: Fitting sample: Sample_12
#> ℹ [2024-08-04 14:44:55.092979]: Fitting sample: Sample_13
#> ℹ [2024-08-04 14:44:55.094443]: Fitting sample: Sample_14
#> ℹ [2024-08-04 14:44:55.095847]: Fitting sample: Sample_15
#> ℹ [2024-08-04 14:44:55.097273]: Fitting sample: Sample_16
#> ℹ [2024-08-04 14:44:55.098704]: Fitting sample: Sample_17
#> ℹ [2024-08-04 14:44:55.112848]: Fitting sample: Sample_18
#> ℹ [2024-08-04 14:44:55.114683]: Fitting sample: Sample_19
#> ℹ [2024-08-04 14:44:55.116206]: Fitting sample: Sample_20
#> ℹ [2024-08-04 14:44:55.117744]: Fitting sample: Sample_21
#> ℹ [2024-08-04 14:44:55.119179]: Fitting sample: Sample_22
#> ℹ [2024-08-04 14:44:55.120603]: Fitting sample: Sample_23
#> ℹ [2024-08-04 14:44:55.122049]: Fitting sample: Sample_24
#> ℹ [2024-08-04 14:44:55.123453]: Fitting sample: Sample_25
#> ℹ [2024-08-04 14:44:55.124861]: Fitting sample: Sample_26
#> ℹ [2024-08-04 14:44:55.126327]: Fitting sample: Sample_27
#> ℹ [2024-08-04 14:44:55.127834]: Fitting sample: Sample_28
#> ℹ [2024-08-04 14:44:55.129334]: Fitting sample: Sample_29
#> ℹ [2024-08-04 14:44:55.130771]: Fitting sample: Sample_30
#> ✔ [2024-08-04 14:44:55.132214]: Done.
#> ℹ [2024-08-04 14:44:55.133623]: Generating output signature exposures.
#> ✔ [2024-08-04 14:44:55.143862]: Done.
#> ℹ [2024-08-04 14:44:55.145636]: Calculating errors (Frobenius Norm).
#> ✔ [2024-08-04 14:44:55.149075]: Done.
#> ℹ [2024-08-04 14:44:55.150635]: 0.098 secs elapsed.
#> ℹ [2024-08-04 14:44:55.153729]: Getting bootstrap exposures (&errors/similarity) for different methods.
#> ℹ [2024-08-04 14:44:55.155108]: This step is time consuming, please be patient.
#> ℹ [2024-08-04 14:44:55.156688]: Processing sample `Sample_1`.
#> ℹ [2024-08-04 14:44:56.4114]: Processing sample `Sample_2`.
#> ℹ [2024-08-04 14:44:57.638795]: Processing sample `Sample_3`.
#> ℹ [2024-08-04 14:44:58.86842]: Processing sample `Sample_4`.
#> ℹ [2024-08-04 14:45:00.087878]: Processing sample `Sample_5`.
#> ℹ [2024-08-04 14:45:01.286896]: Processing sample `Sample_6`.
#> ℹ [2024-08-04 14:45:02.499486]: Processing sample `Sample_7`.
#> ℹ [2024-08-04 14:45:03.735693]: Processing sample `Sample_8`.
#> ℹ [2024-08-04 14:45:04.960866]: Processing sample `Sample_9`.
#> ℹ [2024-08-04 14:45:06.194856]: Processing sample `Sample_10`.
#> ℹ [2024-08-04 14:45:07.409737]: Processing sample `Sample_11`.
#> ℹ [2024-08-04 14:45:08.641541]: Processing sample `Sample_12`.
#> ℹ [2024-08-04 14:45:09.906223]: Processing sample `Sample_13`.
#> ℹ [2024-08-04 14:45:11.115732]: Processing sample `Sample_14`.
#> ℹ [2024-08-04 14:45:12.340082]: Processing sample `Sample_15`.
#> ℹ [2024-08-04 14:45:13.585026]: Processing sample `Sample_16`.
#> ℹ [2024-08-04 14:45:14.820554]: Processing sample `Sample_17`.
#> ℹ [2024-08-04 14:45:16.069739]: Processing sample `Sample_18`.
#> ℹ [2024-08-04 14:45:17.280588]: Processing sample `Sample_19`.
#> ℹ [2024-08-04 14:45:18.526391]: Processing sample `Sample_20`.
#> ℹ [2024-08-04 14:45:19.738187]: Processing sample `Sample_21`.
#> ℹ [2024-08-04 14:45:20.963606]: Processing sample `Sample_22`.
#> ℹ [2024-08-04 14:45:22.208664]: Processing sample `Sample_23`.
#> ℹ [2024-08-04 14:45:23.428511]: Processing sample `Sample_24`.
#> ℹ [2024-08-04 14:45:24.647201]: Processing sample `Sample_25`.
#> ℹ [2024-08-04 14:45:25.826907]: Processing sample `Sample_26`.
#> ℹ [2024-08-04 14:45:27.041701]: Processing sample `Sample_27`.
#> ℹ [2024-08-04 14:45:28.24841]: Processing sample `Sample_28`.
#> ℹ [2024-08-04 14:45:29.434642]: Processing sample `Sample_29`.
#> ℹ [2024-08-04 14:45:30.639001]: Processing sample `Sample_30`.
#> ✔ [2024-08-04 14:45:31.816016]: Gotten.
#> ℹ [2024-08-04 14:45:31.819061]: Reporting p values...
#> ℹ [2024-08-04 14:45:31.820556]: Started.
#> ✔ [2024-08-04 14:45:31.821921]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.824018]: Done.
#> ℹ [2024-08-04 14:45:31.82544]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.826825]: Started.
#> ✔ [2024-08-04 14:45:31.82814]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.830077]: Done.
#> ℹ [2024-08-04 14:45:31.831462]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.832837]: Started.
#> ✔ [2024-08-04 14:45:31.834209]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.836162]: Done.
#> ℹ [2024-08-04 14:45:31.837582]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.838942]: Started.
#> ✔ [2024-08-04 14:45:31.840256]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.842173]: Done.
#> ℹ [2024-08-04 14:45:31.843547]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.844902]: Started.
#> ✔ [2024-08-04 14:45:31.846228]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.848115]: Done.
#> ℹ [2024-08-04 14:45:31.849501]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.85084]: Started.
#> ✔ [2024-08-04 14:45:31.852138]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.854082]: Done.
#> ℹ [2024-08-04 14:45:31.855449]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.856799]: Started.
#> ✔ [2024-08-04 14:45:31.858155]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.86006]: Done.
#> ℹ [2024-08-04 14:45:31.861465]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.862826]: Started.
#> ✔ [2024-08-04 14:45:31.86415]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.866085]: Done.
#> ℹ [2024-08-04 14:45:31.867466]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.868823]: Started.
#> ✔ [2024-08-04 14:45:31.870197]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.872109]: Done.
#> ℹ [2024-08-04 14:45:31.873514]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.874883]: Started.
#> ✔ [2024-08-04 14:45:31.876205]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.878142]: Done.
#> ℹ [2024-08-04 14:45:31.879516]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.880871]: Started.
#> ✔ [2024-08-04 14:45:31.882213]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.884115]: Done.
#> ℹ [2024-08-04 14:45:31.885532]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.887006]: Started.
#> ✔ [2024-08-04 14:45:31.888328]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.89028]: Done.
#> ℹ [2024-08-04 14:45:31.891669]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.893039]: Started.
#> ✔ [2024-08-04 14:45:31.894387]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.896305]: Done.
#> ℹ [2024-08-04 14:45:31.897707]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.899118]: Started.
#> ✔ [2024-08-04 14:45:31.90046]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.902481]: Done.
#> ℹ [2024-08-04 14:45:31.903884]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.905326]: Started.
#> ✔ [2024-08-04 14:45:31.906655]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.908584]: Done.
#> ℹ [2024-08-04 14:45:31.910005]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.911392]: Started.
#> ✔ [2024-08-04 14:45:31.912724]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.914767]: Done.
#> ℹ [2024-08-04 14:45:31.916166]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.917597]: Started.
#> ✔ [2024-08-04 14:45:31.918942]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.920908]: Done.
#> ℹ [2024-08-04 14:45:31.922384]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.923793]: Started.
#> ✔ [2024-08-04 14:45:31.925142]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.927151]: Done.
#> ℹ [2024-08-04 14:45:31.928658]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.93008]: Started.
#> ✔ [2024-08-04 14:45:31.931405]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.933471]: Done.
#> ℹ [2024-08-04 14:45:31.934874]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.936253]: Started.
#> ✔ [2024-08-04 14:45:31.937603]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.93971]: Done.
#> ℹ [2024-08-04 14:45:31.941174]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.942584]: Started.
#> ✔ [2024-08-04 14:45:31.94392]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.945878]: Done.
#> ℹ [2024-08-04 14:45:31.947266]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.948639]: Started.
#> ✔ [2024-08-04 14:45:31.949984]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.951904]: Done.
#> ℹ [2024-08-04 14:45:31.953308]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.95468]: Started.
#> ✔ [2024-08-04 14:45:31.955999]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.957923]: Done.
#> ℹ [2024-08-04 14:45:31.959299]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.960668]: Started.
#> ✔ [2024-08-04 14:45:31.962027]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.963947]: Done.
#> ℹ [2024-08-04 14:45:31.96534]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.966704]: Started.
#> ✔ [2024-08-04 14:45:31.968023]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.970081]: Done.
#> ℹ [2024-08-04 14:45:31.97148]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.972867]: Started.
#> ✔ [2024-08-04 14:45:31.974236]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.97619]: Done.
#> ℹ [2024-08-04 14:45:31.977601]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.978967]: Started.
#> ✔ [2024-08-04 14:45:31.980282]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.982231]: Done.
#> ℹ [2024-08-04 14:45:31.983677]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.985048]: Started.
#> ✔ [2024-08-04 14:45:31.986399]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.988312]: Done.
#> ℹ [2024-08-04 14:45:31.989714]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.991078]: Started.
#> ✔ [2024-08-04 14:45:31.992393]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:31.994317]: Done.
#> ℹ [2024-08-04 14:45:31.995693]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:31.997046]: Started.
#> ✔ [2024-08-04 14:45:31.998395]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:32.000357]: Done.
#> ℹ [2024-08-04 14:45:32.001786]: 0.005 secs elapsed.
#> ✔ [2024-08-04 14:45:32.003173]: Done.
#> ℹ [2024-08-04 14:45:32.004507]: Cleaning results...
#> ✔ [2024-08-04 14:45:32.24305]: Outputing.
#> ℹ [2024-08-04 14:45:32.245404]: Total 37.201 secs elapsed.
rv
#> $expo
#> method sample sig exposure type
#> <char> <char> <fctr> <num> <char>
#> 1: QP Sample_1 SBS1 0.00000 optimal
#> 2: QP Sample_2 SBS1 1115.51740 optimal
#> 3: QP Sample_3 SBS1 1181.67357 optimal
#> 4: QP Sample_4 SBS1 0.00000 optimal
#> 5: QP Sample_5 SBS1 27.23392 optimal
#> ---
#> 1646: QP Sample_30 SBS1 230.74630 Rep_10
#> 1647: QP Sample_30 SBS5 2062.72035 Rep_10
#> 1648: QP Sample_30 SBS9 625.53335 Rep_10
#> 1649: QP Sample_30 SBS2 0.00000 Rep_10
#> 1650: QP Sample_30 SBS13 0.00000 Rep_10
#>
#> $error
#> method sample errors type
#> <char> <char> <num> <char>
#> 1: QP Sample_1 2679.128 optimal
#> 2: QP Sample_2 702.209 optimal
#> 3: QP Sample_3 542.401 optimal
#> 4: QP Sample_4 431.054 optimal
#> 5: QP Sample_5 132.440 optimal
#> ---
#> 326: QP Sample_30 184.349 Rep_6
#> 327: QP Sample_30 185.102 Rep_7
#> 328: QP Sample_30 184.294 Rep_8
#> 329: QP Sample_30 184.066 Rep_9
#> 330: QP Sample_30 184.032 Rep_10
#>
#> $cosine
#> method sample cosine type
#> <char> <char> <num> <char>
#> 1: QP Sample_1 0.870282 optimal
#> 2: QP Sample_2 0.982053 optimal
#> 3: QP Sample_3 0.950683 optimal
#> 4: QP Sample_4 0.977040 optimal
#> 5: QP Sample_5 0.986924 optimal
#> ---
#> 326: QP Sample_30 0.885355 Rep_6
#> 327: QP Sample_30 0.864371 Rep_7
#> 328: QP Sample_30 0.865994 Rep_8
#> 329: QP Sample_30 0.868274 Rep_9
#> 330: QP Sample_30 0.878272 Rep_10
#>
#> $p_val
#> sample method threshold sig p_value
#> <char> <char> <char> <char> <num>
#> 1: Sample_1 QP 0.05 SBS1 1.000000e+00
#> 2: Sample_1 QP 0.05 SBS5 2.070854e-23
#> 3: Sample_1 QP 0.05 SBS9 2.441497e-16
#> 4: Sample_1 QP 0.05 SBS2 1.000000e+00
#> 5: Sample_1 QP 0.05 SBS13 1.000000e+00
#> ---
#> 146: Sample_30 QP 0.05 SBS1 2.906989e-07
#> 147: Sample_30 QP 0.05 SBS5 2.195250e-19
#> 148: Sample_30 QP 0.05 SBS9 1.485679e-12
#> 149: Sample_30 QP 0.05 SBS2 1.000000e+00
#> 150: Sample_30 QP 0.05 SBS13 1.000000e+00
#>
# }
# For general purpose --------------------
W <- matrix(c(1, 2, 3, 4, 5, 6), ncol = 2)
colnames(W) <- c("sig1", "sig2")
W <- apply(W, 2, function(x) x / sum(x))
H <- matrix(c(2, 5, 3, 6, 1, 9, 1, 2), ncol = 4)
colnames(H) <- paste0("samp", 1:4)
V <- W %*% H
V
#> samp1 samp2 samp3 samp4
#> [1,] 1.666667 2.1 2.566667 0.7
#> [2,] 2.333333 3.0 3.333333 1.0
#> [3,] 3.000000 3.9 4.100000 1.3
if (requireNamespace("quadprog")) {
z10 <- sig_fit_bootstrap_batch(V, sig = W, n = 10)
z10
}
#> ℹ [2024-08-04 14:45:32.259043]: Batch Bootstrap Signature Exposure Analysis Started.
#> ℹ [2024-08-04 14:45:32.260548]: Samples to be filtered out:
#> ℹ [2024-08-04 14:45:32.261974]: Finding optimal exposures (&errors) for different methods.
#> ℹ [2024-08-04 14:45:32.263342]: Calling method `QP`.
#> ℹ [2024-08-04 14:45:32.266633]: Started.
#> ℹ [2024-08-04 14:45:32.268]: Signature index not detected.
#> ✔ [2024-08-04 14:45:32.269404]: Signature matrix/data.frame detected.
#> ✔ [2024-08-04 14:45:32.27077]: Database and index checked.
#> ✔ [2024-08-04 14:45:32.272228]: Signature normalized.
#> ℹ [2024-08-04 14:45:32.273594]: Checking row number for catalog matrix and signature matrix.
#> ✔ [2024-08-04 14:45:32.274928]: Checked.
#> ✔ [2024-08-04 14:45:32.276249]: Method 'QP' detected.
#> ✔ [2024-08-04 14:45:32.277626]: Corresponding function generated.
#> ℹ [2024-08-04 14:45:32.278952]: Calling function.
#> ℹ [2024-08-04 14:45:32.280497]: Fitting sample: samp1
#> ℹ [2024-08-04 14:45:32.281994]: Fitting sample: samp2
#> ℹ [2024-08-04 14:45:32.283405]: Fitting sample: samp3
#> ℹ [2024-08-04 14:45:32.284809]: Fitting sample: samp4
#> ✔ [2024-08-04 14:45:32.286237]: Done.
#> ℹ [2024-08-04 14:45:32.287655]: Generating output signature exposures.
#> ✔ [2024-08-04 14:45:32.297174]: Done.
#> ℹ [2024-08-04 14:45:32.298673]: Calculating errors (Frobenius Norm).
#> ✔ [2024-08-04 14:45:32.300193]: Done.
#> ℹ [2024-08-04 14:45:32.301643]: 0.035 secs elapsed.
#> ℹ [2024-08-04 14:45:32.30473]: Getting bootstrap exposures (&errors/similarity) for different methods.
#> ℹ [2024-08-04 14:45:32.306214]: This step is time consuming, please be patient.
#> ℹ [2024-08-04 14:45:32.308008]: Processing sample `samp1`.
#> ℹ [2024-08-04 14:45:33.416123]: Processing sample `samp2`.
#> ℹ [2024-08-04 14:45:34.505133]: Processing sample `samp3`.
#> ℹ [2024-08-04 14:45:35.567932]: Processing sample `samp4`.
#> ✔ [2024-08-04 14:45:36.654435]: Gotten.
#> ℹ [2024-08-04 14:45:36.656861]: Reporting p values...
#> ℹ [2024-08-04 14:45:36.658408]: Started.
#> ✔ [2024-08-04 14:45:36.659771]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:36.661594]: Done.
#> ℹ [2024-08-04 14:45:36.662993]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:36.664399]: Started.
#> ✔ [2024-08-04 14:45:36.665745]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:36.667471]: Done.
#> ℹ [2024-08-04 14:45:36.668859]: 0.004 secs elapsed.
#> ℹ [2024-08-04 14:45:36.670273]: Started.
#> ✔ [2024-08-04 14:45:36.671604]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:36.673507]: Done.
#> ℹ [2024-08-04 14:45:36.674941]: 0.005 secs elapsed.
#> ℹ [2024-08-04 14:45:36.676347]: Started.
#> ✔ [2024-08-04 14:45:36.677701]: Batch mode enabled.
#> ✔ [2024-08-04 14:45:36.679519]: Done.
#> ℹ [2024-08-04 14:45:36.681006]: 0.005 secs elapsed.
#> ✔ [2024-08-04 14:45:36.682461]: Done.
#> ℹ [2024-08-04 14:45:36.683814]: Cleaning results...
#> ✔ [2024-08-04 14:45:36.72216]: Outputing.
#> ℹ [2024-08-04 14:45:36.724425]: Total 4.465 secs elapsed.
#> $expo
#> method sample sig exposure type
#> <char> <char> <fctr> <num> <char>
#> 1: QP samp1 sig1 2.000000 optimal
#> 2: QP samp2 sig1 3.000000 optimal
#> 3: QP samp3 sig1 1.000000 optimal
#> 4: QP samp4 sig1 1.000000 optimal
#> 5: QP samp1 sig2 5.000000 optimal
#> 6: QP samp2 sig2 6.000000 optimal
#> 7: QP samp3 sig2 9.000000 optimal
#> 8: QP samp4 sig2 2.000000 optimal
#> 9: QP samp1 sig1 0.333333 Rep_1
#> 10: QP samp1 sig2 6.666667 Rep_1
#> 11: QP samp1 sig1 0.000000 Rep_2
#> 12: QP samp1 sig2 7.000000 Rep_2
#> 13: QP samp1 sig1 7.000000 Rep_3
#> 14: QP samp1 sig2 0.000000 Rep_3
#> 15: QP samp1 sig1 0.000000 Rep_4
#> 16: QP samp1 sig2 7.000000 Rep_4
#> 17: QP samp1 sig1 0.000000 Rep_5
#> 18: QP samp1 sig2 7.000000 Rep_5
#> 19: QP samp1 sig1 0.000000 Rep_6
#> 20: QP samp1 sig2 7.000000 Rep_6
#> 21: QP samp1 sig1 5.333333 Rep_7
#> 22: QP samp1 sig2 1.666667 Rep_7
#> 23: QP samp1 sig1 0.333333 Rep_8
#> 24: QP samp1 sig2 6.666667 Rep_8
#> 25: QP samp1 sig1 7.000000 Rep_9
#> 26: QP samp1 sig2 0.000000 Rep_9
#> 27: QP samp1 sig1 7.000000 Rep_10
#> 28: QP samp1 sig2 0.000000 Rep_10
#> 29: QP samp2 sig1 0.000000 Rep_1
#> 30: QP samp2 sig2 9.000000 Rep_1
#> 31: QP samp2 sig1 9.000000 Rep_2
#> 32: QP samp2 sig2 0.000000 Rep_2
#> 33: QP samp2 sig1 9.000000 Rep_3
#> 34: QP samp2 sig2 0.000000 Rep_3
#> 35: QP samp2 sig1 9.000000 Rep_4
#> 36: QP samp2 sig2 0.000000 Rep_4
#> 37: QP samp2 sig1 0.000000 Rep_5
#> 38: QP samp2 sig2 9.000000 Rep_5
#> 39: QP samp2 sig1 0.000000 Rep_6
#> 40: QP samp2 sig2 9.000000 Rep_6
#> 41: QP samp2 sig1 0.000000 Rep_7
#> 42: QP samp2 sig2 9.000000 Rep_7
#> 43: QP samp2 sig1 0.000000 Rep_8
#> 44: QP samp2 sig2 9.000000 Rep_8
#> 45: QP samp2 sig1 0.000000 Rep_9
#> 46: QP samp2 sig2 9.000000 Rep_9
#> 47: QP samp2 sig1 9.000000 Rep_10
#> 48: QP samp2 sig2 0.000000 Rep_10
#> 49: QP samp3 sig1 0.000000 Rep_1
#> 50: QP samp3 sig2 10.000000 Rep_1
#> 51: QP samp3 sig1 0.000000 Rep_2
#> 52: QP samp3 sig2 10.000000 Rep_2
#> 53: QP samp3 sig1 0.000000 Rep_3
#> 54: QP samp3 sig2 10.000000 Rep_3
#> 55: QP samp3 sig1 3.333333 Rep_4
#> 56: QP samp3 sig2 6.666667 Rep_4
#> 57: QP samp3 sig1 8.333333 Rep_5
#> 58: QP samp3 sig2 1.666667 Rep_5
#> 59: QP samp3 sig1 0.000000 Rep_6
#> 60: QP samp3 sig2 10.000000 Rep_6
#> 61: QP samp3 sig1 8.333333 Rep_7
#> 62: QP samp3 sig2 1.666667 Rep_7
#> 63: QP samp3 sig1 0.000000 Rep_8
#> 64: QP samp3 sig2 10.000000 Rep_8
#> 65: QP samp3 sig1 3.333333 Rep_9
#> 66: QP samp3 sig2 6.666667 Rep_9
#> 67: QP samp3 sig1 10.000000 Rep_10
#> 68: QP samp3 sig2 0.000000 Rep_10
#> 69: QP samp4 sig1 3.000000 Rep_1
#> 70: QP samp4 sig2 0.000000 Rep_1
#> 71: QP samp4 sig1 3.000000 Rep_2
#> 72: QP samp4 sig2 0.000000 Rep_2
#> 73: QP samp4 sig1 3.000000 Rep_3
#> 74: QP samp4 sig2 0.000000 Rep_3
#> 75: QP samp4 sig1 3.000000 Rep_4
#> 76: QP samp4 sig2 0.000000 Rep_4
#> 77: QP samp4 sig1 3.000000 Rep_5
#> 78: QP samp4 sig2 0.000000 Rep_5
#> 79: QP samp4 sig1 0.000000 Rep_6
#> 80: QP samp4 sig2 3.000000 Rep_6
#> 81: QP samp4 sig1 0.000000 Rep_7
#> 82: QP samp4 sig2 3.000000 Rep_7
#> 83: QP samp4 sig1 3.000000 Rep_8
#> 84: QP samp4 sig2 0.000000 Rep_8
#> 85: QP samp4 sig1 3.000000 Rep_9
#> 86: QP samp4 sig2 0.000000 Rep_9
#> 87: QP samp4 sig1 0.000000 Rep_10
#> 88: QP samp4 sig2 3.000000 Rep_10
#> method sample sig exposure type
#>
#> $error
#> method sample errors type
#> <char> <char> <num> <char>
#> 1: QP samp1 0.000 optimal
#> 2: QP samp2 0.000 optimal
#> 3: QP samp3 0.000 optimal
#> 4: QP samp4 0.000 optimal
#> 5: QP samp1 0.236 Rep_1
#> 6: QP samp1 0.283 Rep_2
#> 7: QP samp1 0.707 Rep_3
#> 8: QP samp1 0.283 Rep_4
#> 9: QP samp1 0.283 Rep_5
#> 10: QP samp1 0.283 Rep_6
#> 11: QP samp1 0.471 Rep_7
#> 12: QP samp1 0.236 Rep_8
#> 13: QP samp1 0.707 Rep_9
#> 14: QP samp1 0.707 Rep_10
#> 15: QP samp2 0.424 Rep_1
#> 16: QP samp2 0.849 Rep_2
#> 17: QP samp2 0.849 Rep_3
#> 18: QP samp2 0.849 Rep_4
#> 19: QP samp2 0.424 Rep_5
#> 20: QP samp2 0.424 Rep_6
#> 21: QP samp2 0.424 Rep_7
#> 22: QP samp2 0.424 Rep_8
#> 23: QP samp2 0.424 Rep_9
#> 24: QP samp2 0.849 Rep_10
#> 25: QP samp3 0.141 Rep_1
#> 26: QP samp3 0.141 Rep_2
#> 27: QP samp3 0.141 Rep_3
#> 28: QP samp3 0.330 Rep_4
#> 29: QP samp3 1.037 Rep_5
#> 30: QP samp3 0.141 Rep_6
#> 31: QP samp3 1.037 Rep_7
#> 32: QP samp3 0.141 Rep_8
#> 33: QP samp3 0.330 Rep_9
#> 34: QP samp3 1.273 Rep_10
#> 35: QP samp4 0.283 Rep_1
#> 36: QP samp4 0.283 Rep_2
#> 37: QP samp4 0.283 Rep_3
#> 38: QP samp4 0.283 Rep_4
#> 39: QP samp4 0.283 Rep_5
#> 40: QP samp4 0.141 Rep_6
#> 41: QP samp4 0.141 Rep_7
#> 42: QP samp4 0.283 Rep_8
#> 43: QP samp4 0.283 Rep_9
#> 44: QP samp4 0.141 Rep_10
#> method sample errors type
#>
#> $cosine
#> method sample cosine type
#> <char> <char> <num> <char>
#> 1: QP samp1 1.000000 optimal
#> 2: QP samp2 1.000000 optimal
#> 3: QP samp3 1.000000 optimal
#> 4: QP samp4 1.000000 optimal
#> 5: QP samp1 0.895314 Rep_1
#> 6: QP samp1 0.795784 Rep_2
#> 7: QP samp1 0.925820 Rep_3
#> 8: QP samp1 0.915052 Rep_4
#> 9: QP samp1 0.634858 Rep_5
#> 10: QP samp1 0.967382 Rep_6
#> 11: QP samp1 0.982299 Rep_7
#> 12: QP samp1 0.995086 Rep_8
#> 13: QP samp1 0.908688 Rep_9
#> 14: QP samp1 0.991460 Rep_10
#> 15: QP samp2 0.909964 Rep_1
#> 16: QP samp2 0.960001 Rep_2
#> 17: QP samp2 0.977008 Rep_3
#> 18: QP samp2 0.993859 Rep_4
#> 19: QP samp2 0.973450 Rep_5
#> 20: QP samp2 0.986928 Rep_6
#> 21: QP samp2 0.828303 Rep_7
#> 22: QP samp2 0.892710 Rep_8
#> 23: QP samp2 0.973450 Rep_9
#> 24: QP samp2 0.918262 Rep_10
#> 25: QP samp3 0.682355 Rep_1
#> 26: QP samp3 0.887369 Rep_2
#> 27: QP samp3 0.996748 Rep_3
#> 28: QP samp3 0.964274 Rep_4
#> 29: QP samp3 0.949101 Rep_5
#> 30: QP samp3 0.949671 Rep_6
#> 31: QP samp3 0.997805 Rep_7
#> 32: QP samp3 0.949671 Rep_8
#> 33: QP samp3 0.964274 Rep_9
#> 34: QP samp3 0.944911 Rep_10
#> 35: QP samp4 0.956183 Rep_1
#> 36: QP samp4 0.956183 Rep_2
#> 37: QP samp4 0.956183 Rep_3
#> 38: QP samp4 0.801784 Rep_4
#> 39: QP samp4 0.836660 Rep_5
#> 40: QP samp4 0.986928 Rep_6
#> 41: QP samp4 0.713506 Rep_7
#> 42: QP samp4 0.836660 Rep_8
#> 43: QP samp4 0.836660 Rep_9
#> 44: QP samp4 0.713506 Rep_10
#> method sample cosine type
#>
#> $p_val
#> sample method threshold sig p_value
#> <char> <char> <char> <char> <num>
#> 1: samp1 QP 0.05 sig1 0.0277068123
#> 2: samp1 QP 0.05 sig2 0.0024736045
#> 3: samp2 QP 0.05 sig1 0.0303471317
#> 4: samp2 QP 0.05 sig2 0.0041413035
#> 5: samp3 QP 0.05 sig1 0.0279108829
#> 6: samp3 QP 0.05 sig2 0.0005029653
#> 7: samp4 QP 0.05 sig1 0.0010629845
#> 8: samp4 QP 0.05 sig2 0.0680670588
#>