Code

Backfills & deprecating legacy tables#10

Code

Merged

max-ostapenko merged 59 commits intomainHTTPArchive/dataform:mainfrom fiscal-owlHTTPArchive/dataform:fiscal-owlCopy head branch name to clipboardNov 1, 2024

+2,130-524

Commits

Commits on Sep 19, 2024

• deprecated

  max-ostapenko

  committed
  99fc4b6

• backfill draft

  max-ostapenko

  committed
  0dbb96c

• cleanup

  max-ostapenko

  committed
  1c30ac3

• null placeholders

  max-ostapenko

  committed
  78e4a23

• sql fix

  max-ostapenko

  committed
  6738785

• fix month range

  max-ostapenko

  committed
  4d69fc6

• literal table names

  max-ostapenko

  committed
  52a8eec

Commits on Sep 27, 2024

• backfill tested

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  a7b7a53

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  6514d89

• dates reset

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  96fee15

• requests_summary

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  a82927c

Commits on Sep 29, 2024

• requests backfill for mid month

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  c504878

• remove legacy pipelines

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  9dc4cf0

• checked against new schema

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  c316e25

• adjusted to a new schema

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  e3cf47b

• backfill_pages

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  8832ffe

Commits on Sep 30, 2024

• legacy removed

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  06d6cb4

• remove legacy datasets

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  9ba236d

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  a57df2d

• metrics sorted

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  57da6fb

• parse features

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  3683a89

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  c55adb6

Commits on Oct 14, 2024

• Merge branch 'main' into fiscal-owl

  max-ostapenko

  committed
  6866120

• lint

  max-ostapenko

  committed
  2870012

• jscpd off

  max-ostapenko

  committed
  992802f

• update js variable names

  max-ostapenko

  committed
  23c29b9

• other cm format

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  14b9585

• Merge branch 'fiscal-owl' into fiscal-owl

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  b176ee3

Commits on Oct 18, 2024

• Merge branch 'main' into fiscal-owl

  max-ostapenko

  authored
  4138af1

Commits on Oct 19, 2024

• pages completed

  max-ostapenko

  committed
  4cafc6f

• summary_pages completed

  max-ostapenko

  committed
  d1dfd49

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  23a522d

• Merge branch 'fiscal-owl' into fiscal-owl

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  3940d6a

Commits on Oct 20, 2024

• without other headers

  max-ostapenko

  committed
  1244e95

• Merge branch 'fiscal-owl' of https://github.com/HTTPArchive/dataform into fiscal-owl

  max-ostapenko

  committed
  4179197

• fix

  max-ostapenko

  committed
  e55d8b4

• fix

  max-ostapenko

  committed
  c7afc11

• fix

  max-ostapenko

  committed
  e03a353

• actual reprocessing queries

  max-ostapenko

  committed
  4a6101a

• fix

  max-ostapenko

  committed
  4eb39ae

• requests complete

  max-ostapenko

  committed
  8d54b1b

• fix casts

  max-ostapenko

  committed
  a38efe0

• wptid from summary

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  86fff73

• Update definitions/output/all/backfill_requests.js

  max-ostapenko

  authored
  e030acc

• Merge branch 'fiscal-owl' into fiscal-owl

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  d9ce5fb

Commits on Oct 21, 2024

• summary update

  max-ostapenko

  committed
  c8e2343

• only valid other headers

  max-ostapenko

  committed
  6032434

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  bc0a104

• Merge branch 'fiscal-owl' into fiscal-owl

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  193027e

• move tables

  max-ostapenko

  committed
  e61df0a

• fix json parsing

  max-ostapenko

  committed
  b2e7b7d

Commits on Oct 22, 2024

• fix summary metrics

  max-ostapenko

  committed
  14816f8

• crawl pipeline updated

  max-ostapenko

  committed
  2898c82

• update dependents

  max-ostapenko

  committed
  d94ca11

Commits on Nov 1, 2024

• response_bodies adjustment

  max-ostapenko

  committed
  01101db

• Merge branch 'main' into main

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  47ebb36

• Merge branch 'fiscal-owl' into fiscal-owl

  

  max-ostapenko

  authored and

  GCP Dataform

  committed
  ff5b06e

• Merge branch 'fiscal-owl' of https://github.com/HTTPArchive/dataform into fiscal-owl

  max-ostapenko

  committed
  530ecd4

• lint

  max-ostapenko

  committed
  5ac19db