How to modify source files?

[jatcwang]

Sometimes we want to modify the actual source files (e.g. https://github.com/sbt/sbt-header). What's the best, principled way to handle something like this in mill?

[lefou]

As Mill is all about caching and avoiding unnecessary work, you will likely want to implement it as a target, not as command. I you want to edit your sources, you need to depend on the sources target.

def updateHeaders1: T[Seq[PathRef]] = T {
  val changed = for {
    file <- Lib.findSourceFiles(sources(), Seq("scala"))
  } yield editFile(file)
  changed.map(PathRef(_))
}

As we changed the files, a next invokation of updateHeaders1 will re-run. To avoid that, we could use a persistent target, remember the checksums of the edited files and only act on changed files.

def updateHeaders2: T[Seq[PathRef]] = T.persistent {
  val previous: Map[os.Path, PathRef] = 
    upickle.default.read[Seq[PathRef]](
      os.read(T.dest / "previous.json")
    )
    .map(pr => pr.path -> pr)
    .toMap
  val changed = for {
    file <- Lib.findSourceFiles(sources(), Seq("scala")
    pr = PathRef(file)
    if previous.get(file).sig != pr.sig
  } yield editFile(file)
  val result = changed.map { file => 
    val pr = PathRef(file)
    previous.updated(file, pr)
    pr
  }
  os.write.over(
    T.dest / "previous.json",
    upickle.default.write(previous.values)
  )
  result
}

Alternatively, you could implement a target checkHeader that checks for proper headers and returns the invalid files, so we could depend on that targets and just change that files.

def checkHeader: T[(Seq[PathRef], Seq[PathRef])] = T {
  val (good, bad) = Lib.findSourceFiles(sources(), Seq("scala").partition(isHeaderOk(_))
  (good.map(PathRef(_), bad.map(PathRef(_))
}

def onlyBadHeaders = T {
  checkHeader()._2
}

def updateHeaders3: T[Seq[PathRef]] = T {
  val (_, files) = onlyBadHeaders()
  val changed = files.map(editFile(_.path))
  changed.map(PathRef(_))
}

[lefou]

It's all pseudo-code, I haven't compiled / tested it!

[jatcwang]

Thanks for all the pointers! 😄