[2023-11-09 08:35:23,238] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:35:23,241] INFO: NVIDIA A10
[2023-11-09 08:35:23,242] INFO: NVIDIA A10
[2023-11-09 08:35:42,874] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-01-29.595745.pth
[2023-11-09 08:35:43,962] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:35:45,673] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 08:35:45,883] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:35:59,189] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:35:59,636] INFO: NVIDIA A10
[2023-11-09 08:35:59,875] INFO: NVIDIA A10
[2023-11-09 08:36:14,297] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:36:14,425] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 08:36:14,668] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 08:36:15,122] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-12-26.926265.pth
[2023-11-09 08:36:16,130] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:36:17,244] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:36:17,760] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:36:18,782] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:36:19,169] INFO: NVIDIA A10
[2023-11-09 08:36:19,420] INFO: NVIDIA A10
[2023-11-09 08:36:22,297] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:36:22,963] INFO: NVIDIA A10
[2023-11-09 08:36:23,304] INFO: NVIDIA A10
[2023-11-09 08:36:31,436] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-18-26.634629.pth
[2023-11-09 08:36:32,953] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:36:33,298] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 08:36:33,548] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:36:35,413] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-24-26.481708.pth
[2023-11-09 08:36:36,443] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:36:37,235] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:36:37,881] INFO: NVIDIA A10
ms_pf_qcd_high_pt, 500
[2023-11-09 08:36:38,218] INFO: NVIDIA A10
 cms_pf_qcd_high_pt
[2023-11-09 08:36:44,003] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-30-26.405813.pth
[2023-11-09 08:36:44,918] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:36:46,450] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 08:36:47,418] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:36:49,176] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:36:50,221] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:36:58,878] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-36-26.356485.pth
[2023-11-09 08:36:59,972] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:37:01,829] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 08:37:02,791] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:37:09,883] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:37:16,229] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:37:19,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:37:21,352] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:37:23,632] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:37:34,302] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:37:42,476] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:37:52,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:37:53,547] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:37:56,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:38:01,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:38:05,550] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:38:12,315] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:38:23,985] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:38:25,059] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:38:30,585] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:38:36,415] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:38:45,073] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:38:46,627] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:38:56,577] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:38:57,375] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:39:03,774] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:39:08,242] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:39:17,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:39:26,599] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:39:29,728] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:39:32,522] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:39:36,203] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:39:40,386] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:39:49,542] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:39:56,424] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:40:02,956] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:40:07,781] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:40:12,903] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:40:20,450] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:40:20,997] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:40:24,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:40:34,318] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:40:38,061] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:40:43,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:40:52,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:40:59,006] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:41:05,246] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:41:07,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:41:13,166] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:41:21,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:41:27,872] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:41:32,404] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 08:41:32,535] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 08:41:32,686] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 08:41:34,909] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:41:42,084] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:41:42,968] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:41:43,238] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-06-27.636442.pth
[2023-11-09 08:41:44,385] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 08:41:44,914] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 08:41:45,354] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 08:41:50,580] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:41:56,180] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:41:59,401] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:42:11,281] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:42:12,776] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:42:16,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:42:25,305] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:42:25,625] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 08:42:26,392] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:42:34,430] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:42:40,946] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:42:45,013] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:42:45,480] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:42:53,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:42:57,037] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:43:09,981] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 08:43:12,654] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:43:13,268] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:43:16,363] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:43:17,030] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:43:24,521] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:43:39,795] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:43:42,338] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:43:44,017] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:43:53,115] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 08:43:53,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:43:55,604] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:44:04,874] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:44:07,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:44:12,526] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:44:13,828] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:44:24,492] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:44:27,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:44:38,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 08:44:38,585] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:44:43,942] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:44:46,181] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:44:46,458] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:44:56,848] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:44:58,644] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:45:09,787] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:45:15,160] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:45:18,626] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:45:23,495] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 08:45:26,233] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:45:27,539] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:45:29,803] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:45:41,914] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:45:46,829] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:45:51,196] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:45:58,867] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:45:59,102] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:46:07,562] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 08:46:08,927] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:46:13,626] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:46:18,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:46:21,022] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:46:28,283] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:46:28,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:46:42,761] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:46:46,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:46:48,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:46:52,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 08:46:55,650] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:47:01,179] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:47:02,022] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:47:16,211] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:47:19,535] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:47:27,240] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:47:30,529] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:47:31,446] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:47:31,880] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:47:35,237] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 08:47:46,370] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:47:49,020] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:48:00,732] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:48:02,883] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:48:05,473] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:48:15,922] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:48:16,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 08:48:19,458] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:48:21,982] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:48:30,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:48:34,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:48:35,496] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:48:50,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:48:50,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:49:00,442] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:49:04,321] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:49:04,543] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 08:49:05,885] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:49:08,712] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:49:24,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:49:24,895] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:49:32,345] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:49:38,716] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:49:42,714] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:49:45,434] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 08:49:58,426] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:49:58,941] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:50:03,391] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 08:50:10,582] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:50:14,356] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:50:22,560] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:50:28,450] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 08:50:29,294] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:50:30,937] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:50:32,902] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:50:39,530] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:50:43,093] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:50:56,563] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:50:58,836] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:50:59,127] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:51:07,648] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 08:51:09,768] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:51:12,124] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 08:51:16,915] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 08:51:28,695] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:51:29,107] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 08:51:32,660] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 08:51:43,530] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:51:46,837] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:51:47,222] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 08:51:53,344] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:51:59,033] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 08:51:59,408] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:52:03,175] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:52:10,618] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:52:12,580] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:52:25,232] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:52:28,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 08:52:29,546] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 08:52:29,618] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:52:40,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:52:42,437] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:52:42,770] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:52:55,183] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:52:59,796] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 08:53:00,081] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:53:12,469] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 08:53:13,079] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 08:53:13,917] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 08:53:20,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:53:25,665] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 08:53:29,286] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 08:53:30,888] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 08:53:42,293] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 08:53:44,601] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 08:53:54,091] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 08:53:57,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 08:53:58,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 08:54:00,786] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 08:54:07,701] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:54:14,046] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 08:54:16,755] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 08:54:24,228] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 08:54:29,570] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 08:54:32,260] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 08:54:41,389] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 08:54:44,965] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 08:54:47,562] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 08:54:53,268] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 08:54:54,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:55:02,393] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 08:55:13,280] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 08:55:16,242] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 08:55:21,104] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 08:55:21,192] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 08:55:30,273] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 08:55:33,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 08:55:39,490] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 08:55:44,851] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 08:55:48,354] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 08:55:51,668] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 08:56:01,261] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 08:56:04,928] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 08:56:07,941] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 08:56:16,330] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 08:56:19,796] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 08:56:21,088] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 08:56:22,201] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 08:56:32,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 08:56:36,305] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 08:56:48,104] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 08:56:51,483] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 08:56:53,547] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 08:56:53,746] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 08:57:04,152] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 08:57:06,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 08:57:19,587] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 08:57:25,304] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 08:57:27,288] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 08:57:35,806] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 08:57:35,973] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 08:57:38,076] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 08:57:50,019] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 08:57:51,274] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 08:57:56,656] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 08:58:00,400] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 08:58:03,857] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 08:58:08,727] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 08:58:18,707] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 08:58:22,102] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 08:58:27,744] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 08:58:28,108] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 08:58:31,801] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 08:58:33,137] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 08:58:41,009] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 08:58:52,099] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 08:58:57,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 08:59:01,694] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 08:59:03,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 08:59:05,908] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 08:59:10,998] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 08:59:11,516] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 08:59:20,587] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 08:59:25,705] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 08:59:31,905] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 08:59:33,200] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 08:59:39,253] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 08:59:49,479] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 08:59:52,877] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 08:59:53,881] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 08:59:54,739] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 09:00:01,067] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 09:00:03,766] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:00:07,616] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 09:00:20,085] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:00:23,501] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:00:32,049] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:00:32,528] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:00:37,191] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:00:38,900] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 09:00:52,048] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:00:54,537] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:01:00,563] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-01/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:01:00,878] INFO: Time taken to make predictions on device 0 is: 25.25 min
[2023-11-09 09:01:01,207] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 09:01:05,675] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:01:08,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:01:17,246] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 09:01:21,946] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:01:24,640] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:01:26,191] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 09:01:37,175] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:01:39,711] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:01:51,767] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-24/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:01:53,376] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:02:04,485] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 09:02:07,368] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:02:08,639] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:02:11,035] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 09:02:20,347] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-36/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:02:20,885] INFO: Time taken to make predictions on device 0 is: 25.29 min
[2023-11-09 09:02:21,419] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 09:02:37,108] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:02:37,371] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-30/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:02:37,614] INFO: Time taken to make predictions on device 0 is: 25.83 min
[2023-11-09 09:02:37,953] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 09:02:47,244] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 09:02:52,065] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 09:03:24,981] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 09:03:38,267] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 09:04:08,702] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 09:04:24,861] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 09:04:51,604] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 09:05:11,626] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 09:05:33,850] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 09:05:58,395] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 09:06:18,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 09:06:46,338] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 09:07:00,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 09:07:33,798] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 09:07:41,662] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 09:08:16,800] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 09:08:26,978] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 09:08:57,683] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 09:09:12,282] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 09:09:40,177] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 09:09:57,563] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 09:10:25,471] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:10:42,596] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 09:11:13,541] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:11:29,336] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 09:11:59,493] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:12:14,022] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 09:12:43,613] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:12:57,124] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 09:13:26,309] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-18/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:13:26,365] INFO: Time taken to make predictions on device 0 is: 36.88 min
[2023-11-09 09:13:26,518] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 09:13:37,236] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 09:14:18,402] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 09:15:02,814] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 09:15:50,535] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 09:16:34,989] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 09:17:18,230] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 09:17:59,843] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-06/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 09:17:59,844] INFO: Time taken to make predictions on device 0 is: 36.24 min
[2023-11-09 09:17:59,846] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:20:24,855] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:25,069] INFO: NVIDIA A10
[2023-11-09 13:20:25,069] INFO: NVIDIA A10
[2023-11-09 13:20:35,768] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:36,118] INFO: NVIDIA A10
[2023-11-09 13:20:36,612] INFO: NVIDIA A10
[2023-11-09 13:20:40,085] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-02-28.902398.pth
[2023-11-09 13:20:40,925] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:20:42,169] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:20:42,529] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:20:42,937] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:43,357] INFO: NVIDIA A10
[2023-11-09 13:20:43,597] INFO: NVIDIA A10
[2023-11-09 13:20:50,741] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:50,894] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 13:20:51,005] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 13:20:51,532] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:52,176] INFO: NVIDIA A10
rom /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-04-28.305056.pth
[2023-11-09 13:20:52,449] INFO: NVIDIA A10
[2023-11-09 13:20:52,997] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:20:54,226] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:20:54,533] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:20:55,412] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:55,951] INFO: NVIDIA A10
[2023-11-09 13:20:56,330] INFO: NVIDIA A10
[2023-11-09 13:20:58,012] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:20:58,489] INFO: NVIDIA A10
[2023-11-09 13:20:58,719] INFO: NVIDIA A10
[2023-11-09 13:21:00,220] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-03-28.528851.pth
[2023-11-09 13:21:01,179] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:02,039] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
-cms-physical_20231107_105813_656942//checkpoints/checkpoint-11-26.993889.pth
[2023-11-09 13:21:02,833] INFO: NVIDIA A10
ms_pf_qcd_high_pt, 500
[2023-11-09 13:21:03,145] INFO: NVIDIA A10
[2023-11-09 13:21:03,452] INFO: Running predictions on cms_pf_qcd_high_pt

    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:03,864] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:04,173] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:21:04,301] INFO: NVIDIA GeForce GTX 1080 Ti
_pt
[2023-11-09 13:21:04,614] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 13:21:10,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:11,761] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-05-28.066963.pth
[2023-11-09 13:21:12,658] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:13,383] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:21:13,867] INFO: NVIDIA A10
[2023-11-09 13:21:14,112] INFO: NVIDIA A10
[2023-11-09 13:21:14,662] INFO: test_dataset: cms_pf_qcd_high_pt, 500
_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-07-27.432857.pth
[2023-11-09 13:21:15,051] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:15,633] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
: Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:15,886] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-09-27.201911.pth
[2023-11-09 13:21:16,383] INFO: NVIDIA A10
[2023-11-09 13:21:16,806] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:17,291] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:18,585] INFO: test_dataset: cms_pf_qcd_high_pt, 500
_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-13-26.853375.pth
[2023-11-09 13:21:19,277] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:20,269] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:20,703] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:20,985] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:21,183] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:21:21,820] INFO: NVIDIA A10
[2023-11-09 13:21:22,205] INFO: NVIDIA A10
[2023-11-09 13:21:22,568] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-08-27.308326.pth
[2023-11-09 13:21:23,671] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:24,698] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:25,015] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:25,688] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:29,331] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-10-27.066493.pth
[2023-11-09 13:21:30,480] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:31,598] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:32,159] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:32,464] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-16-26.719572.pth
[2023-11-09 13:21:33,419] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
: Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:33,720] INFO: NVIDIA A10
[2023-11-09 13:21:34,243] INFO: NVIDIA A10
[2023-11-09 13:21:34,698] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:35,247] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:41,180] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:21:42,453] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-17-26.675206.pth
[2023-11-09 13:21:43,449] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:44,742] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:44,837] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:45,125] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:45,676] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:21:47,470] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:49,567] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:53,366] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-19-26.610014.pth
[2023-11-09 13:21:54,350] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:21:54,965] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:21:55,354] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:21:56,004] INFO: NVIDIA A10
[2023-11-09 13:21:56,352] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:21:57,142] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:21:57,960] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:00,779] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:01,005] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:06,040] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:07,698] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:08,418] INFO: NVIDIA A10
[2023-11-09 13:22:08,734] INFO: NVIDIA A10
[2023-11-09 13:22:10,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:15,344] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-20-26.581604.pth
[2023-11-09 13:22:16,147] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:16,379] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:22:17,228] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:18,137] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:22:19,277] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:22:20,391] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:22,569] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:24,133] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:24,134] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:24,677] INFO: NVIDIA A10
[2023-11-09 13:22:25,012] INFO: NVIDIA A10
[2023-11-09 13:22:25,589] INFO: NVIDIA A10
[2023-11-09 13:22:25,930] INFO: NVIDIA A10
[2023-11-09 13:22:26,528] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:28,739] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:29,211] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:29,588] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-21-26.550251.pth
[2023-11-09 13:22:30,489] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:30,685] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:22:32,515] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:22:33,077] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:22:38,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:38,640] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:38,993] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:39,585] INFO: NVIDIA A10
[2023-11-09 13:22:39,810] INFO: NVIDIA A10
[2023-11-09 13:22:40,510] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:22:44,404] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:45,482] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-25-26.471792.pth
[2023-11-09 13:22:46,351] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-23-26.511038.pth
[2023-11-09 13:22:46,556] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:22:46,619] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:47,281] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:22:48,163] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:22:48,523] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:48,696] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:22:48,999] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:22:49,900] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:22:50,360] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:50,973] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:51,819] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:22:53,389] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:22:53,975] INFO: NVIDIA A10
[2023-11-09 13:22:54,294] INFO: NVIDIA A10
[2023-11-09 13:22:55,780] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-27-26.446815.pth
[2023-11-09 13:22:56,561] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
: Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:22:57,212] INFO: NVIDIA A10
[2023-11-09 13:22:57,551] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:22:57,885] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:22:58,452] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:22:58,863] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-26-26.447519.pth
[2023-11-09 13:22:59,565] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:59,710] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:22:59,908] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:01,595] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:23:02,105] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:23:02,791] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:23:03,434] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:03,743] INFO: NVIDIA A10
[2023-11-09 13:23:04,438] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:08,207] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:23:08,503] INFO: NVIDIA A10
[2023-11-09 13:23:08,927] INFO: NVIDIA A10
[2023-11-09 13:23:10,028] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:10,331] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:23:12,606] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:14,626] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-28-26.435258.pth
[2023-11-09 13:23:15,657] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:17,487] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:23:18,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:18,889] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:19,373] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:20,039] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:23:20,982] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:21,060] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:23:21,295] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:21,747] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:23:22,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:24,334] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-31-26.393447.pth
[2023-11-09 13:23:25,353] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:26,952] INFO: test_dataset: cms_pf_qcd_high_pt, 500
200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:27,413] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:23:27,729] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-32-26.377587.pth
_0.parquet
[2023-11-09 13:23:28,812] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:29,493] INFO: test_dataset: cms_pf_qcd_high_pt, 500
200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:30,020] INFO: Running predictions on cms_pf_qcd_high_pt
PF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:23:31,618] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:34,145] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:35,295] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:23:40,052] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-33-26.378664.pth
_5.parquet
[2023-11-09 13:23:40,951] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:23:41,490] INFO: test_dataset: cms_pf_qcd_high_pt, 500
200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:42,019] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:23:42,721] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:48,888] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:23:49,544] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:23:50,122] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:23:50,535] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:50,648] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:52,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:52,333] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:23:54,140] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:56,976] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:57,412] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:23:57,584] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:23:57,966] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:23:58,262] INFO: NVIDIA A10
[2023-11-09 13:23:58,747] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:23:59,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:00,493] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:24:00,929] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:02,187] INFO: NVIDIA A10
[2023-11-09 13:24:03,428] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:05,005] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:24:05,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:08,888] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:24:09,222] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:24:10,976] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-37-26.348497.pth
_3.parquet
[2023-11-09 13:24:11,944] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:24:12,515] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-38-26.344454.pth
[2023-11-09 13:24:13,546] INFO: test_dataset: cms_pf_qcd_high_pt, 500
200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_4.parquet
: LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:24:13,940] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:24:14,207] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:24:14,477] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:24:15,654] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:18,971] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-35-26.348680.pth
[2023-11-09 13:24:20,024] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_2.parquet
: LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:24:20,309] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:20,621] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:24:21,159] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:24:21,565] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:22,226] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:22,933] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:23,167] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:25,835] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:24:26,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:24:27,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:29,973] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:24:32,665] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:24:35,170] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:35,519] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:35,854] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:24:37,399] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:24:42,375] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:24:42,861] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:45,204] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:45,987] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:24:47,715] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:51,237] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:24:51,669] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:24:52,077] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:24:52,593] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:24:53,330] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:24:54,300] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:55,773] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:24:56,218] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:56,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:24:58,653] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:24:59,319] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:00,989] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:25:03,054] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:04,327] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:25:05,135] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:25:06,193] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:13,033] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:25:15,678] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:17,218] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:25:19,812] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:20,998] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:22,008] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:22,522] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:23,559] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:23,936] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:24,193] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:25:25,284] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:25,650] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:25:26,462] INFO: NVIDIA A10
[2023-11-09 13:25:27,061] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:27,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:28,109] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:29,120] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:25:29,421] INFO: NVIDIA A10
[2023-11-09 13:25:29,833] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:25:31,438] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:33,562] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:33,832] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:25:36,186] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:36,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:25:37,574] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:38,640] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:25:38,953] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:42,743] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:25:45,399] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-22-26.530985.pth
_7.parquet
[2023-11-09 13:25:46,314] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:25:47,803] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:25:48,174] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:25:48,588] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:25:48,748] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-34-26.365953.pth
[2023-11-09 13:25:49,724] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:25:50,107] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:25:50,686] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:25:51,057] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:51,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:25:52,088] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:25:53,305] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:25:53,999] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:54,021] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:25:54,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:25:55,437] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:25:58,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:25:59,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:03,039] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_4.parquet

[2023-11-09 13:26:03,772] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:26:04,415] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:05,970] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:26:07,534] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:09,317] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:11,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:13,397] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:26:14,928] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:17,560] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:19,452] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:19,748] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:26:20,554] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:20,977] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:26:21,935] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:26:22,067] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-09 13:26:22,181] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-09 13:26:22,312] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:22,595] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:24,157] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:24,319] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:24,771] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:26:25,548] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:26,293] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:27,432] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:30,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:31,105] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:32,060] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:26:32,272] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:26:32,836] INFO: NVIDIA A10
pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:26:33,161] INFO: NVIDIA A10
[2023-11-09 13:26:33,980] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:26:34,335] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:26:34,583] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:26:36,285] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:36,851] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:26:37,260] INFO: NVIDIA A10
[2023-11-09 13:26:37,736] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:26:38,820] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:40,141] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:44,478] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:26:48,970] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:50,090] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:50,733] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_1.parquet

[2023-11-09 13:26:50,896] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-40-26.326670.pth
[2023-11-09 13:26:51,320] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:26:51,701] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:26:52,596] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:26:53,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:26:53,462] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:26:53,875] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:26:54,080] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:26:54,535] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:26:54,947] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-39-26.329519.pth
_10.parquet
[2023-11-09 13:26:55,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:26:55,777] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:26:56,108] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-09 13:26:56,757] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:26:56,957] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:26:58,174] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:26:59,756] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:27:00,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:27:02,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:04,140] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:04,580] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:07,064] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:07,657] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:27:08,887] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:27:10,028] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:11,642] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:27:12,838] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-09 13:27:12,968] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 13:27:13,132] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-09 13:27:14,292] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:27:16,049] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:18,939] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:19,328] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:20,067] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:21,570] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:21,792] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:22,918] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:27:22,918] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:27:23,822] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:27:24,261] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:24,426] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:24,921] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:27:26,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:26,879] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:27,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_8.parquet
: LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-09 13:27:27,379] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:27:28,087] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-09 13:27:28,251] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:27:29,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:35,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:37,083] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:37,739] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:38,256] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:39,961] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:27:44,480] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:27:45,175] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:45,535] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:45,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_8.parquet

[2023-11-09 13:27:49,921] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:27:50,883] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:27:51,726] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_1.parquet

[2023-11-09 13:27:51,727] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:27:52,313] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-09 13:27:52,529] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:27:53,131] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:27:53,517] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:54,718] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:27:55,044] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:27:55,899] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:27:56,343] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_9.parquet

[2023-11-09 13:27:57,998] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_9.parquet

[2023-11-09 13:27:58,161] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:27:59,586] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:28:04,679] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:05,257] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:05,803] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:28:09,289] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-09 13:28:09,731] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:28:11,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:12,693] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_9.parquet

[2023-11-09 13:28:13,609] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:28:16,194] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:17,179] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:28:18,815] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:19,563] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:28:20,016] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:28:22,216] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:24,416] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:24,640] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:25,920] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:28:26,586] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:27,512] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_4.parquet

[2023-11-09 13:28:27,813] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:28:31,832] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:28:32,755] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:28:33,456] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:35,788] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:28:37,599] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:41,011] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:28:41,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:28:41,790] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:42,000] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:28:43,832] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:44,389] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:28:45,044] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:28:46,184] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:48,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:28:48,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:28:49,231] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:28:51,032] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:28:51,760] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:28:52,373] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:54,438] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:28:54,915] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:28:55,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:28:56,172] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:28:56,274] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:28:57,639] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:28:58,252] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:28:59,418] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:28:59,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:29:01,772] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:02,729] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:04,655] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:05,163] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:05,516] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:10,555] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:11,861] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:12,972] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:13,205] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:29:13,400] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:13,842] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:29:14,189] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:15,628] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:17,820] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_4.parquet

[2023-11-09 13:29:18,003] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:19,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:21,750] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:29:22,593] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:23,354] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:23,791] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:27,314] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:29:27,726] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:27,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:29:29,394] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:29:29,988] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:29:30,414] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:29:30,510] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:29:33,014] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:33,863] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:35,503] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:37,168] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:39,728] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-09 13:29:39,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:40,725] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:29:41,494] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:41,908] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:43,082] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:29:44,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:29:44,555] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:45,435] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:46,380] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:29:46,490] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:29:47,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:29:48,761] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:29:49,088] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:29:50,007] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:29:50,287] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:29:51,659] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:29:54,335] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:29:55,843] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:57,324] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:29:58,240] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:29:59,475] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:29:59,683] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:30:00,265] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:00,887] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:30:01,597] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:30:02,664] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:30:02,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:05,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:30:08,371] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:09,057] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:10,920] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:30:11,150] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:12,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:12,988] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:14,317] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:30:14,798] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:30:16,063] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:30:17,202] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:18,659] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:30:19,070] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:30:19,369] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:30:20,249] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:21,919] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:25,248] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-09 13:30:26,917] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:30:27,604] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:27,778] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:29,822] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:29,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:30:29,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:30:32,191] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:30:34,199] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:30:36,045] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:36,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:36,518] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:36,899] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:38,092] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:30:38,793] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:30:40,861] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:30:41,270] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:30:41,270] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:30:43,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:30:43,990] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:30:46,696] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:30:47,847] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:48,446] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:48,651] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:51,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:30:52,369] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:55,091] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:30:56,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:30:57,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:30:58,560] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:30:58,894] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:03,971] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:31:04,259] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:31:04,624] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:31:05,326] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:31:05,978] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:31:06,320] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:06,953] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:31:07,410] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:31:07,576] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:31:08,953] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:31:09,273] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:31:11,080] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:12,087] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-09 13:31:14,878] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:31:15,136] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:31:15,506] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:31:17,119] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:17,727] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:18,505] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:20,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:31:20,673] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:31:22,770] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:23,870] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:31:25,612] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:31:26,343] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:28,228] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:31:29,193] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:32,644] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:34,297] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:31:34,509] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:35,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:31:36,393] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:31:37,217] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:31:37,704] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:31:38,654] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:31:39,042] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:31:39,417] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:41,318] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:31:41,897] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:31:43,631] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:31:45,607] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:31:46,463] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:48,243] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:48,629] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:48,841] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:31:49,064] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:50,533] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:51,100] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:31:53,194] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:31:53,654] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-09 13:31:53,832] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:54,228] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:31:55,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:31:57,025] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-09 13:31:57,913] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:31:58,374] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:31:59,417] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:02,450] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:32:02,726] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:32:04,379] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:32:04,679] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:32:05,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:05,624] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:32:07,011] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:32:08,353] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:08,525] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:09,089] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:09,564] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:32:13,039] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:32:14,689] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:14,934] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:32:15,359] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:18,595] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:32:20,902] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:21,853] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:32:23,296] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:23,800] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:26,493] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:27,957] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:32:28,381] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:32:29,444] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:32:30,769] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:32:32,719] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:32:33,228] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:32:33,807] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:32:34,983] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:32:36,138] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:32:37,846] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:40,581] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:41,254] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:32:42,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:32:43,922] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:44,389] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:32:45,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-09 13:32:46,550] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:46,939] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:32:47,136] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:48,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:32:49,826] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:32:51,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:32:52,553] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:54,710] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:32:55,103] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:32:56,489] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:32:57,644] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:32:57,697] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:33:00,057] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:01,050] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:33:01,061] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:33:01,533] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:33:05,311] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:06,111] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:33:07,032] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:09,024] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:33:11,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:33:12,537] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:13,216] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:13,776] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:33:14,358] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:14,559] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:14,884] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:15,243] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:17,955] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:33:18,204] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:19,746] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:21,981] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:22,321] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:22,826] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:33:22,988] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:33:24,385] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:25,647] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:33:25,837] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:26,141] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:28,302] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:33:31,495] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:33:32,159] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:33:32,606] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:37,052] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:37,472] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:38,634] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:39,235] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:33:39,989] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:33:43,793] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:33:44,174] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:45,405] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:45,860] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:46,043] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:33:46,341] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:48,138] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:49,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:50,458] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:33:50,627] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:33:50,931] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:33:51,548] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-09 13:33:52,552] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:53,089] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:53,544] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:33:53,879] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:33:54,454] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:33:55,530] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:33:56,745] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:34:00,021] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:34:02,474] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:34:02,507] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:34:06,244] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:34:06,932] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:34:08,734] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:34:09,570] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:34:10,288] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-09 13:34:10,589] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:34:10,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:11,252] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:34:12,901] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:34:13,069] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:14,707] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:14,963] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:34:18,289] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:34:18,549] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:34:19,345] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:34:21,780] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:23,476] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:34:25,230] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:26,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:34:26,242] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:27,273] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:29,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:34:29,711] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:31,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:34:32,142] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:33,037] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:34:33,424] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:34:34,712] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:39,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:40,300] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:34:40,481] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:34:41,508] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:42,019] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:42,229] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:34:44,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:44,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:44,705] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:46,179] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:34:46,311] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:34:46,894] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:34:48,489] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:34:51,464] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:34:52,897] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:34:53,508] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:34:54,199] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:34:56,175] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:34:58,455] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:34:58,967] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-09 13:35:00,635] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:35:01,605] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:02,575] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:03,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:35:04,221] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:35:05,146] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:35:05,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:06,761] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:35:10,786] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:35:12,930] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:13,134] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:14,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:35:15,305] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:16,963] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:17,126] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:21,112] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:25,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:35:25,788] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:26,080] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:27,279] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:35:27,699] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:35:28,199] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:29,189] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:35:29,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:31,784] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:35:32,030] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:35:32,497] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:35:33,068] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:35:33,738] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:35:35,263] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:35:37,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:38,409] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:40,109] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:35:40,309] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:35:41,820] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:42,394] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:35:43,161] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:35:44,120] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:35:44,531] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:35:46,255] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:35:47,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:47,363] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:48,226] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:35:50,358] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:35:54,188] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:35:54,423] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:35:56,114] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:35:56,805] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:35:57,194] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:35:57,517] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:35:58,567] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:35:59,268] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:36:00,293] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:02,500] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:36:04,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:36:05,124] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:36:07,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:08,350] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:09,006] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:09,309] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:36:10,051] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:36:10,894] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:36:12,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:13,639] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:13,711] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:14,993] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:36:15,783] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:15,960] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:36:17,546] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:36:21,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:22,401] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:36:24,405] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-09 13:36:25,570] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:26,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:26,409] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:36:26,688] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:29,264] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:29,456] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:29,941] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:30,357] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:36:30,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:36:30,834] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:36:33,275] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:35,239] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:36:35,618] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:36:36,227] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:36:36,838] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:37,255] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:38,749] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:39,423] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:41,376] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:36:43,798] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:36:45,282] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:45,310] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:50,295] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:36:50,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:36:51,487] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:36:53,794] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:36:54,857] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:36:55,169] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:36:56,155] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:36:57,238] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:36:57,961] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:36:58,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:37:00,484] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:37:00,890] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:02,074] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:02,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:37:02,696] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:37:03,355] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-09 13:37:03,712] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:37:06,227] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:37:06,667] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:37:06,735] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:37:07,931] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:08,385] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:37:09,986] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:37:13,009] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:14,385] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:14,785] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:16,947] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:37:17,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:37:20,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:21,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:37:23,428] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:25,308] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:26,195] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:26,366] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:26,572] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:37:27,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:29,075] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:37:29,558] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:31,210] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:37:31,916] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:37:32,226] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:32,721] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:37:33,530] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:34,042] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:37:34,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:37:36,349] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:36,350] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:37,149] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:37:39,217] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:37:40,193] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:40,773] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:41,312] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-09 13:37:42,860] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:37:43,353] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-09 13:37:49,319] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:37:49,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:37:51,859] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:37:52,889] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:37:53,161] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:37:53,652] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:37:55,326] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:37:57,166] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:37:58,412] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:37:58,962] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:38:00,747] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:38:01,220] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:38:01,678] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:38:02,048] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:38:03,084] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:38:03,774] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:38:04,171] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:38:07,851] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:38:08,096] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:38:08,825] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:38:09,727] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:38:10,272] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:38:12,949] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:38:15,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:38:19,375] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:20,604] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:38:21,359] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:38:21,551] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:38:22,298] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:38:22,543] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:38:23,224] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:38:24,923] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:38:25,043] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:38:26,055] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:38:26,656] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-09 13:38:28,439] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:28,761] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:29,552] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:38:31,286] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:38:31,655] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:38:31,827] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:32,167] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:38:32,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:35,820] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:38:36,878] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:38:37,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:38:38,122] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:38:39,212] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:40,016] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:38:46,593] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:38:47,388] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:38:47,662] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:38:48,471] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:38:49,381] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:38:51,358] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:38:52,915] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:38:53,815] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:38:55,256] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:38:58,213] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:38:58,448] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:38:58,659] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:38:58,914] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:38:59,453] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:38:59,589] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:39:00,077] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:01,768] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:39:03,979] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:39:04,940] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:39:05,344] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:39:06,065] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:06,473] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:39:07,925] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:39:08,981] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:39:10,741] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-09 13:39:11,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:39:15,079] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:39:16,155] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:39:16,464] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:39:17,301] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:17,837] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:39:18,141] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:39:18,855] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:39:20,370] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:39:23,814] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:39:24,157] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:39:25,048] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:26,870] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:39:26,989] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:39:27,544] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:29,011] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:29,541] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:30,013] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:39:32,675] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:39:33,833] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:39:34,787] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:39:35,131] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:39:36,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:38,332] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:38,634] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:39:41,498] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:39:45,497] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:45,772] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:39:46,030] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:39:48,015] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:39:48,218] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:39:49,512] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:39:49,752] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:39:53,306] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:53,307] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:39:54,916] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:56,245] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-09 13:39:56,598] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:39:57,009] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:39:58,070] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:58,264] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:39:59,020] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:39:59,378] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:40:00,113] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:40:02,275] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:40:03,688] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:40:06,059] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:40:06,269] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:40:06,862] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:40:07,091] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:40:08,492] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:40:11,955] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:15,254] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:17,163] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:40:17,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:19,263] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:40:19,417] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:40:19,440] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:40:20,589] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:40:20,782] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:21,202] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:21,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:40:21,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:40:24,991] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:25,194] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:25,824] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:27,053] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:40:27,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:40:28,302] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:40:28,740] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:40:29,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:40:30,909] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:40:31,537] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:40:33,545] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:40:34,357] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:34,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:36,260] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:40:36,957] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:36,968] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:37,522] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:40:38,884] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:40:39,565] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-09 13:40:41,626] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:40:44,916] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:40:45,509] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:40:45,500] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:40:47,676] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:40:48,074] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:40:50,456] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:40:50,532] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:40:54,855] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:40:56,312] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:40:56,699] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:40:57,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:40:57,796] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:40:58,215] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:40:59,420] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:41:00,108] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:41:01,458] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:41:03,524] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:41:03,873] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:41:04,631] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:41:04,885] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:41:08,753] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:41:09,008] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:41:10,569] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:13,002] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:41:13,026] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:41:13,934] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:41:14,560] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:15,915] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:41:17,672] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:41:18,032] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:41:19,640] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:20,111] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:20,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-09 13:41:20,912] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:41:22,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:41:24,875] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:25,064] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:41:26,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:41:27,326] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:27,968] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:41:28,627] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:41:28,825] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:31,890] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:41:33,488] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:41:33,806] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:41:34,619] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:34,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:40,384] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:40,495] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:41,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:41:41,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:41:43,273] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:41:43,839] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:41:44,559] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:41:47,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:41:48,123] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:41:48,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:41:49,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:41:53,947] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:54,562] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:41:56,366] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:41:57,412] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:41:57,828] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:41:58,554] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:41:59,432] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:02,880] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:42:03,057] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:42:04,521] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:42:06,051] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:42:06,453] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:42:07,183] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:42:07,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:42:07,768] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-09 13:42:08,760] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:42:08,821] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:42:10,376] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:10,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:12,119] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:12,220] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:15,709] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:16,173] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:42:17,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:42:18,144] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:18,462] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:19,032] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:42:21,302] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:42:23,851] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:42:24,612] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:25,577] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:42:26,733] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:42:27,537] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:42:27,926] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:29,213] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:29,952] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:42:32,621] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:42:34,411] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:35,607] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:42:36,393] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:42:36,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:36,820] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:42:40,283] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:42:42,291] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:42,784] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:43,287] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:42:44,335] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:42:46,976] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:42:48,103] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:42:48,675] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:42:49,005] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:42:51,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-09 13:42:51,904] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:42:53,470] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:42:53,658] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:54,808] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:42:57,398] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:42:58,008] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:42:58,414] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:42:58,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:42:59,571] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:00,245] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:43:01,160] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:43:02,833] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:43:03,441] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:43:03,699] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:05,434] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:05,803] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:43:06,701] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:08,939] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:09,155] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:10,615] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:43:10,702] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:43:16,336] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:17,901] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:18,799] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:43:19,274] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:43:19,583] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:43:21,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:43:25,015] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:43:25,579] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:43:25,771] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:26,002] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:43:26,898] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:43:27,042] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:43:28,146] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:43:28,874] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:43:29,386] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:43:30,139] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:43:30,692] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:32,393] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:43:33,493] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:43:33,738] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:43:34,421] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:35,566] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:43:36,986] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:43:38,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:38,685] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-09 13:43:39,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:43,985] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:43:45,758] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:43:46,728] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:43:47,534] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:43:47,824] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:49,168] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:43:50,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:52,995] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:43:53,529] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:53,916] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:43:55,411] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:43:56,069] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:43:56,620] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:43:56,657] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:43:57,541] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:43:58,532] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:43:59,241] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:43:59,418] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:44:00,683] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:44:01,182] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:44:01,474] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:44:02,806] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:02,999] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:44:03,954] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:04,185] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:44:07,689] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:08,627] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:44:09,512] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:44:09,622] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:44:10,031] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:11,704] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:14,666] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:44:16,713] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:44:18,346] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:44:19,603] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:22,027] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:44:22,867] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:44:23,310] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:44:24,552] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:44:25,014] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:44:25,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:26,363] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:44:27,016] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:44:27,141] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-02/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:44:27,442] INFO: Time taken to make predictions on device 0 is: 23.75 min
[2023-11-09 13:44:28,046] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:44:28,189] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:30,230] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:44:30,678] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:31,344] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:44:31,611] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:44:32,023] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:44:32,840] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:44:35,700] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:44:36,534] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:44:37,707] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:38,982] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:41,007] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:42,333] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:45,103] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:44:46,111] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:44:47,163] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:50,095] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:51,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:44:52,178] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:44:52,530] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:44:52,716] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:44:53,325] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:53,905] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:54,323] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:44:56,944] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:44:57,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:58,844] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:44:59,539] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:00,792] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:45:01,367] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:45:01,930] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:45:03,935] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-09/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:04,621] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:45:05,828] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:06,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:45:06,928] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-09 13:45:08,558] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:11,186] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:45:12,223] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:45:15,479] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:45:17,156] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:45:18,499] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:45:20,113] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:45:20,895] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:45:21,124] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:45:22,666] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:45:23,704] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:45:24,751] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:25,414] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-10/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:45:25,766] INFO: Time taken to make predictions on device 0 is: 23.89 min
[2023-11-09 13:45:25,964] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:45:27,270] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:28,397] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:45:29,661] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:45:30,205] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:45:30,619] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:45:31,039] INFO: Time taken to make predictions on device 0 is: 24.19 min
s_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:45:31,532] INFO: Plotting distributions for cms_pf_qcd_high_pt
ms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:45:32,084] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:45:34,305] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:45:34,448] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:45:35,994] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:45:36,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:45:39,423] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:45:39,711] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:41,112] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:45,335] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:45:46,088] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:45:46,482] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:45:48,065] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:45:48,756] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-07/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:49,495] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:45:50,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-17/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:52,655] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:45:54,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:45:54,908] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-09 13:45:57,268] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:45:57,523] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-04/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:45:57,926] INFO: Time taken to make predictions on device 0 is: 25.05 min
[2023-11-09 13:45:58,207] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:45:58,899] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:45:59,087] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:45:59,541] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:00,344] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:46:01,751] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:46:02,503] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:46:03,236] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:46:04,545] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:06,957] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-05/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:46:07,270] INFO: Time taken to make predictions on device 0 is: 24.86 min
[2023-11-09 13:46:07,564] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:46:08,619] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:09,197] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-19/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:46:09,512] INFO: Time taken to make predictions on device 0 is: 24.23 min
[2023-11-09 13:46:09,837] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:46:15,198] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:16,152] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:46:17,113] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:46:17,228] INFO: Time taken to make predictions on device 0 is: 24.99 min
[2023-11-09 13:46:17,579] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:46:17,732] INFO: Time taken to make predictions on device 0 is: 24.53 min
s_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:46:18,076] INFO: Plotting distributions for cms_pf_qcd_high_pt
ms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:46:19,362] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:46:19,434] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-08/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:46:19,786] INFO: Time taken to make predictions on device 0 is: 24.90 min
[2023-11-09 13:46:20,113] INFO: Plotting distributions for cms_pf_qcd_high_pt
ms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:46:24,610] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:46:26,901] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:27,273] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:27,571] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:27,767] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:46:28,648] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:46:30,370] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:46:31,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:46:32,440] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-33/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:35,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:46:35,999] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:37,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:46:39,872] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-09 13:46:43,163] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-27/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:43,995] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:46:44,109] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:46:48,245] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:46:48,979] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:46:50,211] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:46:54,394] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-21/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:46:54,784] INFO: Time taken to make predictions on device 0 is: 24.36 min
[2023-11-09 13:46:55,099] INFO: Plotting distributions for cms_pf_qcd_high_pt
ms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:46:55,433] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:56,026] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:46:56,221] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:46:58,363] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:46:58,594] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:46:59,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:46:59,612] INFO: Time taken to make predictions on device 0 is: 23.28 min
[2023-11-09 13:47:00,059] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:00,577] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:47:01,262] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:47:03,104] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:47:03,606] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-32/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:47:04,315] INFO: Time taken to make predictions on device 0 is: 23.57 min
[2023-11-09 13:47:04,639] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:09,219] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:47:09,609] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:47:09,707] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:47:10,163] INFO: Time taken to make predictions on device 0 is: 24.19 min
[2023-11-09 13:47:10,372] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:17,886] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:47:18,464] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:47:20,196] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-09 13:47:21,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:47:22,109] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-25/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:47:22,437] INFO: Time taken to make predictions on device 0 is: 24.55 min
[2023-11-09 13:47:22,824] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:23,694] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-20/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:47:24,234] INFO: Time taken to make predictions on device 0 is: 25.07 min
[2023-11-09 13:47:24,546] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:26,275] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:47:28,873] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:47:29,896] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-23/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:47:30,175] INFO: Time taken to make predictions on device 0 is: 24.68 min
[2023-11-09 13:47:30,533] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:47:31,119] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:47:31,611] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:47:32,279] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:47:33,329] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:47:34,949] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:47:35,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:47:38,824] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:47:39,663] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:47:39,891] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:47:45,242] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:47:47,624] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:47:48,117] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:47:50,813] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:47:57,811] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:48:00,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:48:01,725] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:48:02,311] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:48:02,559] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:48:02,654] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:48:04,627] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:48:05,916] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:48:07,732] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-09 13:48:08,277] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-16/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:08,502] INFO: Time taken to make predictions on device 0 is: 26.55 min
[2023-11-09 13:48:08,730] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:48:15,242] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:48:15,527] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-26/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:15,877] INFO: Time taken to make predictions on device 0 is: 25.22 min
[2023-11-09 13:48:16,226] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:48:17,522] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:48:18,189] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:48:24,269] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:48:27,846] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:48:29,870] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-29/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:30,494] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:48:30,645] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:48:31,072] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:48:31,991] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:48:34,376] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:48:36,023] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-03/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:36,271] INFO: Time taken to make predictions on device 0 is: 27.54 min
[2023-11-09 13:48:36,545] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:48:42,250] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:48:46,019] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:48:50,835] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-09 13:48:55,667] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:48:58,015] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:48:58,282] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-28/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:58,559] INFO: Time taken to make predictions on device 0 is: 25.66 min
s_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:48:58,936] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-31/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:48:59,131] INFO: Time taken to make predictions on device 0 is: 25.52 min
[2023-11-09 13:48:59,423] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:00,870] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-38/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:49:01,140] INFO: Time taken to make predictions on device 0 is: 24.77 min
[2023-11-09 13:49:01,505] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:03,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:49:09,047] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:49:11,899] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:49:15,562] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:49:17,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:49:22,697] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-35/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:49:22,942] INFO: Time taken to make predictions on device 0 is: 25.02 min
[2023-11-09 13:49:23,197] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:24,618] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-39/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:49:25,255] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:49:28,747] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-09 13:49:31,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-37/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:49:32,200] INFO: Time taken to make predictions on device 0 is: 25.29 min
[2023-11-09 13:49:32,604] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:37,028] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:49:43,039] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:49:50,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:49:50,461] INFO: Time taken to make predictions on device 0 is: 22.89 min
[2023-11-09 13:49:50,808] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:51,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-40/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:49:51,618] INFO: Time taken to make predictions on device 0 is: 22.95 min
[2023-11-09 13:49:51,939] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:49:52,958] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:50:00,480] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:50:12,579] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-09 13:50:12,876] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:50:14,165] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:50:23,126] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:50:39,609] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:50:41,137] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:50:47,181] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:50:54,552] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:50:54,585] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:50:55,511] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-09 13:51:08,575] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-22/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:51:08,899] INFO: Time taken to make predictions on device 0 is: 25.33 min
[2023-11-09 13:51:09,240] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:51:23,847] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:51:24,260] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-34/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:51:24,590] INFO: Time taken to make predictions on device 0 is: 25.56 min
[2023-11-09 13:51:24,901] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:51:32,474] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:51:35,856] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:51:38,257] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-09 13:52:05,888] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:52:15,195] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:52:17,054] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:52:23,730] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-09 13:52:45,207] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:52:55,131] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:52:57,830] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:53:07,937] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-09 13:53:25,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:53:36,037] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:53:40,671] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:53:49,172] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-09 13:54:09,225] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:54:20,172] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:54:21,384] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:54:34,537] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-09 13:54:55,597] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:55:00,630] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:55:07,688] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:55:20,733] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-09 13:55:37,180] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 13:55:39,336] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:55:52,507] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:56:07,459] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-09 13:56:14,829] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 13:56:22,380] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:56:35,998] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:56:53,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-09 13:56:55,536] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 13:57:04,160] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:57:04,331] INFO: Time taken to make predictions on device 0 is: 35.72 min
[2023-11-09 13:57:04,481] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:57:17,914] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-11/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:57:18,268] INFO: Time taken to make predictions on device 0 is: 36.23 min
[2023-11-09 13:57:18,385] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:57:38,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 13:57:41,060] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-09 13:58:19,546] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 13:58:27,000] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-09 13:58:59,382] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 13:59:11,241] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-09 13:59:38,040] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 13:59:38,199] INFO: Time taken to make predictions on device 0 is: 33.06 min
[2023-11-09 13:59:38,389] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-09 13:59:52,081] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-09 14:00:34,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-09 14:01:20,402] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-09 14:02:08,085] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-09 14:02:53,404] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-09 14:03:37,640] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-09 14:04:19,829] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-15/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-09 14:04:19,830] INFO: Time taken to make predictions on device 0 is: 36.86 min
[2023-11-09 14:04:19,832] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-13 08:37:20,572] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-13 08:37:21,063] INFO: NVIDIA A10
[2023-11-13 08:37:21,063] INFO: NVIDIA A10
[2023-11-13 08:37:37,300] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-60-26.257606.pth
[2023-11-13 08:37:38,228] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-13 08:37:39,704] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-13 08:37:40,576] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-13 08:38:10,029] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-13 08:38:39,601] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-13 08:39:08,624] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-13 08:39:38,800] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-13 08:40:09,288] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-13 08:40:38,272] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-13 08:41:07,496] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-13 08:41:31,196] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-13 08:41:32,020] INFO: NVIDIA A10
[2023-11-13 08:41:32,689] INFO: NVIDIA A10
[2023-11-13 08:41:35,581] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-13 08:41:52,574] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-70-26.236859.pth
[2023-11-13 08:41:53,391] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-13 08:41:55,172] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-13 08:41:56,919] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-13 08:42:03,839] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-13 08:42:20,487] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-13 08:42:20,729] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-13 08:42:21,305] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-13 08:42:27,324] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-13 08:42:35,016] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-13 08:42:35,250] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-13 08:42:35,575] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-13 08:42:36,415] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-13 08:42:37,485] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-99-26.172849.pth
[2023-11-13 08:42:38,676] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-13 08:42:39,440] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-13 08:42:40,546] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-13 08:42:50,640] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-80-26.207721.pth
[2023-11-13 08:42:51,749] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-13 08:42:52,308] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-13 08:42:53,336] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-13 08:42:58,763] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-13 08:43:04,609] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-13 08:43:24,765] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-13 08:43:29,055] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-13 08:43:34,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-13 08:43:36,681] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-13 08:44:00,317] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-13 08:44:00,857] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-13 08:44:12,742] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-13 08:44:24,923] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-13 08:44:27,526] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-13 08:44:31,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-13 08:44:56,561] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-13 08:45:00,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-13 08:45:03,229] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-13 08:45:11,797] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-13 08:45:26,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-13 08:45:34,373] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-13 08:45:49,452] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-13 08:45:56,958] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-13 08:45:58,697] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-13 08:46:04,410] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-13 08:46:26,698] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-13 08:46:32,934] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-13 08:46:39,408] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-13 08:46:47,090] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-13 08:46:54,036] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-13 08:47:07,147] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-13 08:47:25,897] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-13 08:47:27,614] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-13 08:47:33,527] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-13 08:47:36,237] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-13 08:47:54,947] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-13 08:48:07,955] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-13 08:48:16,492] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-13 08:48:21,528] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-13 08:48:27,270] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-13 08:48:34,410] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-13 08:48:55,571] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-13 08:49:02,363] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-13 08:49:04,812] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-13 08:49:26,666] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-13 08:49:33,008] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-13 08:49:46,590] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-13 08:49:47,422] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-13 08:49:59,112] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-13 08:50:03,528] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-13 08:50:29,129] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-13 08:50:33,940] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-13 08:50:35,903] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-13 08:50:39,356] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-13 08:50:55,722] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-13 08:51:04,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-13 08:51:21,089] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-13 08:51:24,168] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-13 08:51:28,208] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-13 08:51:33,480] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-13 08:51:58,095] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-13 08:52:05,584] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-13 08:52:06,209] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-13 08:52:11,751] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-13 08:52:23,197] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-13 08:52:35,611] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-13 08:52:45,502] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-13 08:52:52,313] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-13 08:52:54,353] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-13 08:53:08,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-13 08:53:21,947] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-13 08:53:26,582] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-13 08:53:36,291] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-13 08:53:37,255] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-13 08:53:50,582] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-13 08:54:08,962] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-13 08:54:10,708] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-13 08:54:19,776] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-13 08:54:24,835] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-13 08:54:41,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-13 08:54:47,599] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-13 08:54:57,272] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-13 08:55:12,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-13 08:55:14,757] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-13 08:55:39,510] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-13 08:55:44,298] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-13 08:56:02,066] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-13 08:56:11,238] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-13 08:56:15,101] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-13 08:56:26,931] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-13 08:56:40,661] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-13 08:56:45,380] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-13 08:56:47,118] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-13 08:57:06,340] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-13 08:57:07,314] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-13 08:57:15,963] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-13 08:57:30,766] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-13 08:57:36,700] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-13 08:57:46,902] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-13 08:57:55,125] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-13 08:58:07,016] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-13 08:58:15,914] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-13 08:58:20,563] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-13 08:58:36,362] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-13 08:58:39,366] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-13 08:58:44,416] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-13 08:59:06,872] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-13 08:59:11,690] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-13 08:59:27,924] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-13 08:59:35,021] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-13 08:59:38,815] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-13 08:59:56,515] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-13 09:00:01,637] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-13 09:00:08,181] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-13 09:00:09,623] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-13 09:00:31,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-13 09:00:39,124] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-13 09:00:57,461] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-13 09:01:01,229] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-13 09:01:08,340] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-13 09:01:29,086] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-13 09:01:31,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-13 09:01:36,719] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-13 09:01:44,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-13 09:02:02,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-13 09:02:03,392] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-60/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-13 09:02:04,039] INFO: Time taken to make predictions on device 0 is: 24.39 min
[2023-11-13 09:02:04,922] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-13 09:02:18,911] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-13 09:02:30,067] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-13 09:02:32,171] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-13 09:03:01,548] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-13 09:03:05,678] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-13 09:03:10,525] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-13 09:03:29,906] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-13 09:03:48,091] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-13 09:03:56,837] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-13 09:03:58,796] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-13 09:04:23,473] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-13 09:04:38,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-13 09:04:42,199] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-13 09:04:52,296] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-13 09:05:20,552] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-13 09:05:23,428] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-13 09:05:24,583] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-13 09:05:52,449] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-13 09:06:04,351] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-13 09:06:04,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-13 09:06:20,482] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-13 09:06:46,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-70/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-13 09:06:47,399] INFO: Time taken to make predictions on device 0 is: 24.83 min
[2023-11-13 09:06:48,024] INFO: Plotting distributions for cms_pf_qcd_high_pt
ms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-13 09:06:50,950] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-13 09:07:30,085] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-13 09:07:37,556] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-13 09:08:15,464] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-13 09:08:22,922] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-13 09:08:58,422] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-13 09:09:11,629] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-13 09:09:39,493] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-13 09:09:57,473] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-13 09:10:23,953] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-13 09:10:40,822] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-13 09:11:09,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-13 09:11:28,613] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-13 09:11:55,319] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-13 09:12:16,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-13 09:12:40,639] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-13 09:13:03,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-13 09:13:28,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-13 09:13:52,446] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-13 09:14:13,588] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-13 09:14:42,592] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-13 09:14:57,269] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-13 09:15:30,485] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-13 09:15:38,224] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-13 09:16:16,123] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-13 09:16:19,704] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-13 09:16:59,238] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-13 09:17:05,466] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-13 09:17:43,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-13 09:17:53,361] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-13 09:18:30,106] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-13 09:18:38,541] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-13 09:19:20,755] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-13 09:19:22,942] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-13 09:20:05,766] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-80/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-13 09:20:06,000] INFO: Time taken to make predictions on device 0 is: 37.20 min
[2023-11-13 09:20:06,470] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-13 09:20:07,752] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-13 09:20:54,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-13 09:21:41,710] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-99/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-13 09:21:41,770] INFO: Time taken to make predictions on device 0 is: 39.01 min
[2023-11-13 09:21:41,773] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-14 15:42:17,412] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 15:42:17,592] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-14 15:42:17,592] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-14 15:42:30,177] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-140-26.135630.pth
[2023-11-14 15:42:31,272] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 15:42:31,629] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 15:42:31,647] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 15:43:14,506] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 15:44:01,042] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 15:44:44,934] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-14 15:45:30,288] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 15:46:15,998] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 15:47:01,162] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 15:47:36,974] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 15:47:37,688] INFO: NVIDIA A10
[2023-11-14 15:47:38,004] INFO: NVIDIA A10
[2023-11-14 15:47:46,101] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 15:47:57,481] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-120-26.163478.pth
[2023-11-14 15:47:58,330] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 15:47:59,942] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 15:48:00,580] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 15:48:18,154] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 15:48:18,616] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-14 15:48:18,918] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-14 15:48:29,176] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 15:48:31,373] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 15:48:32,128] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-130-26.147027.pth
[2023-11-14 15:48:33,358] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 15:48:33,778] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 15:48:33,931] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 15:49:02,153] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 15:49:10,339] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 15:49:12,258] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 15:49:32,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-14 15:49:54,360] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 15:49:58,766] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 15:50:03,155] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 15:50:34,142] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 15:50:39,739] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 15:51:05,094] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 15:51:15,348] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 15:51:23,739] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 15:51:34,649] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 15:51:59,356] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 15:52:03,506] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 15:52:04,104] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 15:52:31,160] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 15:52:40,095] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 15:52:44,615] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 15:53:04,424] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 15:53:21,047] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 15:53:27,633] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 15:53:31,534] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 15:54:00,147] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 15:54:01,745] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 15:54:12,496] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 15:54:27,829] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 15:54:38,942] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 15:54:54,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 15:54:56,847] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 15:55:22,768] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 15:55:23,408] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 15:55:39,484] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 15:55:53,201] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 15:56:00,076] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 15:56:19,658] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 15:56:23,304] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 15:56:39,012] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 15:56:52,489] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 15:57:06,975] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 15:57:14,517] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 15:57:20,287] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 15:57:51,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 15:57:52,001] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 15:58:20,447] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-14 15:58:29,979] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 15:58:39,498] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 15:58:52,167] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 15:59:10,703] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 15:59:20,797] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 15:59:21,032] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 15:59:51,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 15:59:52,200] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:00:08,779] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:00:24,494] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:00:30,553] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 16:00:53,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:00:55,748] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:01:07,532] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 16:01:20,698] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:01:41,947] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:01:50,794] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 16:01:51,966] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:02:21,819] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:02:22,524] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:02:30,578] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-14 16:02:47,741] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-14 16:03:11,407] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:03:12,600] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 16:03:17,715] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-14 16:03:46,353] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:03:49,399] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 16:03:54,957] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:04:15,625] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:04:33,612] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:04:45,420] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:05:13,316] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:05:19,568] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:05:40,700] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:06:00,419] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:06:04,097] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:06:11,400] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:06:38,524] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:06:41,615] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:06:47,793] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:07:11,816] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:07:21,917] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:07:33,169] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:07:42,938] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:08:02,159] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:08:13,378] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:08:17,027] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:08:38,955] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-14 16:08:43,760] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:08:58,359] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:09:12,703] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:09:20,737] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-14 16:09:39,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:09:42,558] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:10:03,555] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:10:05,504] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:10:28,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:10:34,220] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:10:44,189] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:11:05,320] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:11:14,475] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:11:26,792] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:11:34,416] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:11:59,409] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:12:02,663] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-14 16:12:07,105] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:12:30,157] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-12/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-14 16:12:30,692] INFO: Time taken to make predictions on device 0 is: 24.50 min
[2023-11-14 16:12:31,316] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-14 16:12:46,408] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:12:48,371] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:13:31,445] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:13:31,687] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:14:14,277] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:14:16,649] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:14:54,459] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:15:01,797] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:15:36,020] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:15:45,895] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:16:21,029] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:16:30,574] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:17:09,227] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:17:14,512] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:17:53,972] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:17:55,890] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:18:33,888] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:18:37,636] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-14 16:19:12,995] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:19:19,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-14/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-14 16:19:19,770] INFO: Time taken to make predictions on device 0 is: 36.80 min
[2023-11-14 16:19:19,880] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-14 16:19:55,376] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:20:40,954] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:21:23,114] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_checkpoint-13/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:22:00,421] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 16:22:00,627] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-14 16:22:00,627] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-14 16:22:16,269] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-110-26.158400.pth
[2023-11-14 16:22:17,488] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 16:22:17,782] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 16:22:17,886] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 16:22:28,755] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 16:22:28,966] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-14 16:22:29,075] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-14 16:22:42,302] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-140-26.135630.pth
[2023-11-14 16:22:43,302] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 16:22:43,596] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 16:22:43,713] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 16:22:53,462] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-14 16:22:54,052] INFO: NVIDIA A10
[2023-11-14 16:22:54,481] INFO: NVIDIA A10
[2023-11-14 16:22:59,051] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 16:23:15,623] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-130-26.147027.pth
[2023-11-14 16:23:16,559] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-14 16:23:18,588] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-14 16:23:18,902] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-14 16:23:21,610] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 16:23:44,144] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 16:23:50,710] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-14 16:24:05,399] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 16:24:20,978] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-14 16:24:27,333] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-14 16:24:48,638] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-14 16:24:51,440] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-14 16:25:12,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 16:25:22,433] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 16:25:33,732] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-14 16:25:53,423] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 16:25:59,117] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 16:26:18,282] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-14 16:26:24,440] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 16:26:42,636] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 16:26:53,388] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 16:27:01,465] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-14 16:27:22,114] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 16:27:27,399] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 16:27:44,030] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-14 16:27:49,750] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 16:28:09,702] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 16:28:23,891] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 16:28:24,536] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-14 16:28:50,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 16:28:51,276] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 16:29:03,564] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-14 16:29:21,512] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 16:29:38,419] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 16:29:47,880] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 16:29:49,660] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-14 16:30:15,033] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 16:30:18,736] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 16:30:28,136] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-14 16:30:44,122] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 16:31:02,283] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 16:31:08,784] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-14 16:31:13,254] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 16:31:42,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 16:31:43,621] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 16:31:46,454] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-14 16:32:13,191] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 16:32:22,038] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 16:32:24,226] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-14 16:32:40,277] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 16:33:12,027] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 16:33:32,741] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 16:33:40,421] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-14 16:33:41,898] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-14 16:34:12,357] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 16:34:14,727] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 16:34:26,411] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-14 16:34:41,092] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 16:34:57,133] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 16:35:11,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-14 16:35:12,529] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:35:35,695] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 16:35:44,645] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:35:53,406] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-14 16:36:14,547] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:36:15,022] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 16:36:33,614] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-14 16:36:40,823] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:36:59,424] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 16:37:11,745] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:37:20,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-14 16:37:40,117] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:37:40,401] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-14 16:38:03,429] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-14 16:38:05,596] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-14 16:38:24,660] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 16:38:34,762] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-14 16:38:49,611] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-14 16:39:02,377] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 16:39:03,833] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:39:30,266] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-14 16:39:32,283] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:39:46,085] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:40:01,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:40:16,098] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-14 16:40:29,413] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:40:30,622] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:40:56,689] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:41:02,772] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-14 16:41:12,781] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:41:26,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:41:47,213] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-14 16:41:52,344] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:41:56,281] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:42:26,092] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:42:27,978] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-14 16:42:36,692] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:42:55,976] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:43:15,011] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-14 16:43:21,239] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:43:26,516] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:43:55,459] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:43:57,469] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-14 16:43:58,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-14 16:44:24,102] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:44:36,315] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-14 16:44:39,841] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-14 16:44:51,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:45:17,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:45:19,544] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-14 16:45:21,799] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:45:46,366] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:46:02,485] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-14 16:46:02,594] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:46:16,659] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:46:43,602] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-14 16:46:44,952] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:46:45,904] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:47:12,816] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-14 16:47:27,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-14 16:47:28,160] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:47:39,770] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-14 16:47:40,107] INFO: Time taken to make predictions on device 0 is: 24.35 min
[2023-11-14 16:47:40,871] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-14 16:48:07,643] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:48:09,706] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-14 16:48:50,172] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-14 16:48:50,542] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:49:34,000] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:49:34,762] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-14 16:50:17,256] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:50:19,478] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-14 16:50:59,833] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:51:04,122] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-14 16:51:44,340] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:51:48,285] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-14 16:52:27,567] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:52:34,682] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-14 16:53:08,674] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:53:19,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-14 16:53:46,496] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:54:02,218] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-14 16:54:26,091] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:54:42,220] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-14 16:55:09,759] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:55:22,975] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-14 16:55:54,960] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:56:07,486] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-14 16:56:36,658] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:56:54,388] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-14 16:57:18,748] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-14 16:57:38,399] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-14 16:57:58,219] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-14 16:57:58,385] INFO: Time taken to make predictions on device 0 is: 35.24 min
[2023-11-14 16:57:58,560] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-14 16:58:21,283] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-14 16:59:03,187] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-14 16:59:03,383] INFO: Time taken to make predictions on device 0 is: 36.76 min
[2023-11-14 16:59:03,438] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-11-30 14:05:37,723] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-30 14:05:38,192] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-30 14:05:38,192] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-11-30 14:05:52,431] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-200-26.102901.pth
[2023-11-30 14:05:53,693] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-30 14:05:54,354] INFO: test_dataset: cms_pf_qcd_high_pt, 500
[2023-11-30 14:05:54,844] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-30 14:06:37,892] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-30 14:07:24,990] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-30 14:07:47,473] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-11-30 14:07:47,762] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-30 14:07:48,111] INFO: NVIDIA GeForce RTX 2080 Ti
[2023-11-30 14:07:58,838] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//checkpoints/checkpoint-200-26.102901.pth
[2023-11-30 14:07:59,786] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-11-30 14:08:00,241] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-11-30 14:08:00,652] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-11-30 14:08:10,200] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-30 14:08:40,720] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-11-30 14:08:58,252] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-30 14:09:21,342] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-11-30 14:09:46,033] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-30 14:10:01,093] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-11-30 14:10:32,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-30 14:10:42,599] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-11-30 14:11:18,865] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-30 14:11:22,022] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-11-30 14:12:04,295] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-30 14:12:04,842] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-11-30 14:12:44,632] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-11-30 14:12:48,033] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-30 14:13:27,174] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-11-30 14:13:39,334] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-30 14:14:09,260] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-11-30 14:14:23,009] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-30 14:14:46,450] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-11-30 14:15:09,025] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-30 14:15:25,070] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-11-30 14:15:50,889] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-30 14:16:02,239] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-11-30 14:16:44,055] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-11-30 14:17:28,388] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-11-30 14:18:04,707] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-11-30 14:18:45,522] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-11-30 14:19:25,845] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-11-30 14:20:03,085] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-11-30 14:20:40,560] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-11-30 14:21:16,385] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-11-30 14:21:54,068] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-11-30 14:22:33,586] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-11-30 14:23:11,659] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-11-30 14:23:52,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-11-30 14:24:32,801] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-11-30 14:25:12,306] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-11-30 14:25:52,431] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-11-30 14:26:32,396] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-11-30 14:27:08,632] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-11-30 14:27:44,974] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-11-30 14:28:19,158] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-11-30 14:28:55,435] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-11-30 14:29:34,564] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-11-30 14:30:13,215] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-11-30 14:30:54,468] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-11-30 14:31:33,314] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-11-30 14:32:11,624] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-11-30 14:32:48,979] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-11-30 14:33:24,040] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-11-30 14:34:03,472] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-11-30 14:34:42,675] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-11-30 14:35:19,962] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-11-30 14:35:56,931] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-11-30 14:36:32,074] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-11-30 14:37:12,514] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-11-30 14:37:52,310] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-11-30 14:38:32,332] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-11-30 14:39:10,959] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-11-30 14:39:44,261] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-11-30 14:40:22,369] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-11-30 14:41:03,196] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_50.parquet
[2023-11-30 14:41:40,550] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_51.parquet
[2023-11-30 14:42:17,631] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_52.parquet
[2023-11-30 14:42:59,505] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_53.parquet
[2023-11-30 14:43:36,791] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_54.parquet
[2023-11-30 14:44:12,844] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_55.parquet
[2023-11-30 14:44:52,041] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_56.parquet
[2023-11-30 14:45:28,617] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_57.parquet
[2023-11-30 14:46:11,211] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_58.parquet
[2023-11-30 14:46:47,849] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_59.parquet
[2023-11-30 14:47:25,543] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_60.parquet
[2023-11-30 14:48:00,533] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_61.parquet
[2023-11-30 14:48:41,425] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_62.parquet
[2023-11-30 14:49:21,092] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_63.parquet
[2023-11-30 14:49:58,575] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_64.parquet
[2023-11-30 14:50:34,844] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_65.parquet
[2023-11-30 14:51:10,782] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_66.parquet
[2023-11-30 14:51:47,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_67.parquet
[2023-11-30 14:52:27,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_68.parquet
[2023-11-30 14:53:04,315] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_69.parquet
[2023-11-30 14:53:45,777] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_70.parquet
[2023-11-30 14:54:23,184] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_71.parquet
[2023-11-30 14:55:02,677] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_72.parquet
[2023-11-30 14:55:41,526] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_73.parquet
[2023-11-30 14:56:18,282] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_74.parquet
[2023-11-30 14:56:57,929] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_75.parquet
[2023-11-30 14:57:34,565] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_76.parquet
[2023-11-30 14:58:13,234] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_77.parquet
[2023-11-30 14:58:52,991] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_78.parquet
[2023-11-30 14:59:32,316] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_79.parquet
[2023-11-30 15:00:12,182] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_80.parquet
[2023-11-30 15:00:49,978] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_81.parquet
[2023-11-30 15:01:28,219] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_82.parquet
[2023-11-30 15:02:03,612] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_83.parquet
[2023-11-30 15:02:39,518] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_84.parquet
[2023-11-30 15:03:15,636] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_85.parquet
[2023-11-30 15:03:51,967] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_86.parquet
[2023-11-30 15:04:34,036] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_87.parquet
[2023-11-30 15:05:07,538] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_88.parquet
[2023-11-30 15:05:48,520] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_89.parquet
[2023-11-30 15:06:24,357] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_90.parquet
[2023-11-30 15:07:05,631] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_91.parquet
[2023-11-30 15:07:46,840] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_92.parquet
[2023-11-30 15:08:26,065] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_93.parquet
[2023-11-30 15:09:02,606] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_94.parquet
[2023-11-30 15:09:41,415] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_95.parquet
[2023-11-30 15:10:19,919] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_96.parquet
[2023-11-30 15:10:55,644] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_97.parquet
[2023-11-30 15:11:27,210] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_98.parquet
[2023-11-30 15:12:04,895] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942//preds_./cms_pf_qcd_high_pt/pred_0_99.parquet
[2023-11-30 15:12:04,895] INFO: Time taken to make predictions on device 0 is: 64.07 min
[2023-11-30 15:12:04,897] INFO: Plotting distributions for cms_pf_qcd_high_pt
[2023-12-13 11:04:43,368] INFO: Will use cpu
[2023-12-13 11:04:46,874] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/checkpoints/checkpoint-200-26.102901.pth
[2023-12-13 11:04:46,874] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/checkpoints/checkpoint-200-26.102901.pth
[2023-12-13 11:04:47,278] INFO: MLPF(
  (nn0): Sequential(
    (0): Linear(in_features=42, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=512, bias=True)
  )
  (conv_id): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (conv_reg): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (nn_id): Sequential(
    (0): Linear(in_features=1578, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=9, bias=True)
  )
  (nn_pt): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_eta): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_phi): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=2, bias=True)
  )
  (nn_energy): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_charge): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=3, bias=True)
  )
)
[2023-12-13 11:04:47,278] INFO: MLPF(
  (nn0): Sequential(
    (0): Linear(in_features=42, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=512, bias=True)
  )
  (conv_id): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (conv_reg): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (nn_id): Sequential(
    (0): Linear(in_features=1578, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=9, bias=True)
  )
  (nn_pt): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_eta): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_phi): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=2, bias=True)
  )
  (nn_energy): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_charge): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=3, bias=True)
  )
)
[2023-12-13 11:04:47,821] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-12-13 11:04:47,821] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-12-13 11:04:47,865] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-12-13 11:04:47,865] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-12-13 11:05:26,214] INFO: Will use single-gpu: NVIDIA GeForce GTX 1080 Ti
[2023-12-13 11:05:33,609] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/checkpoints/checkpoint-200-26.102901.pth
[2023-12-13 11:05:33,609] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/checkpoints/checkpoint-200-26.102901.pth
[2023-12-13 11:05:33,646] INFO: MLPF(
  (nn0): Sequential(
    (0): Linear(in_features=42, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=512, bias=True)
  )
  (conv_id): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (conv_reg): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (nn_id): Sequential(
    (0): Linear(in_features=1578, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=9, bias=True)
  )
  (nn_pt): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_eta): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_phi): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=2, bias=True)
  )
  (nn_energy): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_charge): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=3, bias=True)
  )
)
[2023-12-13 11:05:33,646] INFO: MLPF(
  (nn0): Sequential(
    (0): Linear(in_features=42, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=512, bias=True)
  )
  (conv_id): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (conv_reg): ModuleList(
    (0-2): 3 x GravNetLayer(
      (conv1): GravNetConv(512, 512, k=16)
      (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (dropout): Dropout(p=0.0, inplace=False)
    )
  )
  (nn_id): Sequential(
    (0): Linear(in_features=1578, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=9, bias=True)
  )
  (nn_pt): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_eta): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_phi): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=2, bias=True)
  )
  (nn_energy): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=1, bias=True)
  )
  (nn_charge): Sequential(
    (0): Linear(in_features=1587, out_features=512, bias=True)
    (1): ELU(alpha=1.0)
    (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=512, out_features=3, bias=True)
  )
)
[2023-12-13 11:05:33,740] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-12-13 11:05:33,740] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-12-13 11:05:33,754] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-12-13 11:05:33,754] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-12-13 11:17:24,695] INFO: Will use torch.nn.parallel.DistributedDataParallel() and 2 gpus
[2023-12-13 11:17:26,612] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-12-13 11:17:26,612] INFO: NVIDIA GeForce GTX 1080 Ti
[2023-12-13 11:17:38,880] INFO: Loaded model weights from /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/checkpoints/checkpoint-200-26.102901.pth
[2023-12-13 11:17:39,937] INFO: DistributedDataParallel(
  (module): MLPF(
    (nn0): Sequential(
      (0): Linear(in_features=42, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=512, bias=True)
    )
    (conv_id): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (conv_reg): ModuleList(
      (0-2): 3 x GravNetLayer(
        (conv1): GravNetConv(512, 512, k=16)
        (norm1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.0, inplace=False)
      )
    )
    (nn_id): Sequential(
      (0): Linear(in_features=1578, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=9, bias=True)
    )
    (nn_pt): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_eta): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_phi): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=2, bias=True)
    )
    (nn_energy): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=1, bias=True)
    )
    (nn_charge): Sequential(
      (0): Linear(in_features=1587, out_features=512, bias=True)
      (1): ELU(alpha=1.0)
      (2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
      (3): Dropout(p=0.0, inplace=False)
      (4): Linear(in_features=512, out_features=3, bias=True)
    )
  )
)
[2023-12-13 11:17:40,284] INFO: test_dataset: cms_pf_qcd_high_pt, 1000
[2023-12-13 11:17:40,340] INFO: Running predictions on cms_pf_qcd_high_pt
[2023-12-13 11:18:25,511] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_0.parquet
[2023-12-13 11:19:10,313] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_1.parquet
[2023-12-13 11:19:54,367] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_2.parquet
[2023-12-13 11:20:40,594] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_3.parquet
[2023-12-13 11:21:24,230] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_4.parquet
[2023-12-13 11:22:12,291] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_5.parquet
[2023-12-13 11:22:59,726] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_6.parquet
[2023-12-13 11:23:47,760] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_7.parquet
[2023-12-13 11:24:35,405] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_8.parquet
[2023-12-13 11:25:18,110] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_9.parquet
[2023-12-13 11:26:00,996] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_10.parquet
[2023-12-13 11:26:43,306] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_11.parquet
[2023-12-13 11:27:29,656] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_12.parquet
[2023-12-13 11:28:20,432] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_13.parquet
[2023-12-13 11:29:01,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_14.parquet
[2023-12-13 11:29:47,314] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_15.parquet
[2023-12-13 11:30:32,301] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_16.parquet
[2023-12-13 11:31:13,781] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_17.parquet
[2023-12-13 11:31:55,326] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_18.parquet
[2023-12-13 11:32:33,061] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_19.parquet
[2023-12-13 11:33:15,715] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_20.parquet
[2023-12-13 11:34:00,076] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_21.parquet
[2023-12-13 11:34:43,364] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_22.parquet
[2023-12-13 11:35:29,797] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_23.parquet
[2023-12-13 11:36:14,775] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_24.parquet
[2023-12-13 11:36:59,082] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_25.parquet
[2023-12-13 11:37:43,706] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_26.parquet
[2023-12-13 11:38:28,321] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_27.parquet
[2023-12-13 11:39:08,501] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_28.parquet
[2023-12-13 11:39:48,787] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_29.parquet
[2023-12-13 11:40:26,710] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_30.parquet
[2023-12-13 11:41:07,279] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_31.parquet
[2023-12-13 11:41:51,642] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_32.parquet
[2023-12-13 11:42:33,932] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_33.parquet
[2023-12-13 11:43:20,290] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_34.parquet
[2023-12-13 11:44:03,400] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_35.parquet
[2023-12-13 11:44:46,253] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_36.parquet
[2023-12-13 11:45:28,227] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_37.parquet
[2023-12-13 11:46:07,591] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_38.parquet
[2023-12-13 11:46:51,963] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_39.parquet
[2023-12-13 11:47:36,023] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_40.parquet
[2023-12-13 11:48:17,453] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_41.parquet
[2023-12-13 11:48:58,602] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_42.parquet
[2023-12-13 11:49:37,629] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_43.parquet
[2023-12-13 11:50:23,573] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_44.parquet
[2023-12-13 11:51:07,448] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_45.parquet
[2023-12-13 11:51:51,414] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_46.parquet
[2023-12-13 11:52:34,459] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_47.parquet
[2023-12-13 11:53:12,852] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_48.parquet
[2023-12-13 11:53:55,858] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_49.parquet
[2023-12-13 11:54:41,899] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_50.parquet
[2023-12-13 11:55:24,646] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_51.parquet
[2023-12-13 11:56:06,498] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_52.parquet
[2023-12-13 11:56:52,846] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_53.parquet
[2023-12-13 11:57:34,687] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_54.parquet
[2023-12-13 11:58:15,075] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_55.parquet
[2023-12-13 11:58:59,381] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_56.parquet
[2023-12-13 11:59:40,630] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_57.parquet
[2023-12-13 12:00:27,099] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_58.parquet
[2023-12-13 12:01:09,409] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_59.parquet
[2023-12-13 12:01:50,869] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_60.parquet
[2023-12-13 12:02:30,364] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_61.parquet
[2023-12-13 12:03:16,945] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_62.parquet
[2023-12-13 12:04:01,548] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_63.parquet
[2023-12-13 12:04:44,590] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_64.parquet
[2023-12-13 12:05:25,390] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_65.parquet
[2023-12-13 12:06:06,279] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_66.parquet
[2023-12-13 12:06:47,753] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_67.parquet
[2023-12-13 12:07:34,231] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_68.parquet
[2023-12-13 12:08:15,347] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_69.parquet
[2023-12-13 12:09:01,614] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_70.parquet
[2023-12-13 12:09:43,408] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_71.parquet
[2023-12-13 12:10:28,089] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_72.parquet
[2023-12-13 12:11:11,060] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_73.parquet
[2023-12-13 12:11:51,882] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_74.parquet
[2023-12-13 12:12:36,678] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_75.parquet
[2023-12-13 12:13:18,071] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_76.parquet
[2023-12-13 12:14:01,185] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_77.parquet
[2023-12-13 12:14:46,262] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_78.parquet
[2023-12-13 12:15:39,831] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_79.parquet
[2023-12-13 12:16:25,185] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_80.parquet
[2023-12-13 12:17:07,065] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_81.parquet
[2023-12-13 12:17:49,763] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_82.parquet
[2023-12-13 12:18:29,589] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_83.parquet
[2023-12-13 12:19:09,990] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_84.parquet
[2023-12-13 12:19:50,397] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_85.parquet
[2023-12-13 12:20:31,686] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_86.parquet
[2023-12-13 12:21:19,626] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_87.parquet
[2023-12-13 12:21:56,875] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_88.parquet
[2023-12-13 12:22:43,225] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_89.parquet
[2023-12-13 12:23:23,415] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_90.parquet
[2023-12-13 12:24:10,464] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_91.parquet
[2023-12-13 12:24:56,613] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_92.parquet
[2023-12-13 12:25:40,883] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_93.parquet
[2023-12-13 12:26:23,072] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_94.parquet
[2023-12-13 12:27:06,548] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_95.parquet
[2023-12-13 12:27:49,615] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_96.parquet
[2023-12-13 12:28:29,676] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_97.parquet
[2023-12-13 12:29:05,299] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_98.parquet
[2023-12-13 12:29:48,180] INFO: Saved predictions at /pfvol/experiments/MLPF_cms_200epochs_gravnet_pyg-cms-physical_20231107_105813_656942/preds_checkpoint-200-26.102901.pth/cms_pf_qcd_high_pt/pred_0_99.parquet
[2023-12-13 12:29:48,181] INFO: Time taken to make predictions on device 0 is: 72.13 min
[2023-12-13 12:29:48,183] INFO: Plotting distributions for cms_pf_qcd_high_pt