Skip to content

Commit

Permalink
Restructured the code to follow a more conventional project layout (see
Browse files Browse the repository at this point in the history
  • Loading branch information
hectorcorrea committed Jun 3, 2020
1 parent 5cfc967 commit 8814f03
Show file tree
Hide file tree
Showing 16 changed files with 41 additions and 29 deletions.
7 changes: 3 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
/tmp/*
/brown/*
.DS_Store
# ignore the executables
marcli
marcli.exe
marcli_linux
/cmd/marcli/marcli
/cmd/marcli/marcli.exe
/cmd/marcli/marcli_linux
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ MARC command line (marcli) is a toy project that I am working on to help me deal

The basic idea is to create a program that I can run on our Linux servers to parse our very large MARC files. The goal is to be able to copy a single file to our servers to parse these MARC files, find records that match certain criteria and export them for further review.

This program is heavily inspired by amazing work that Terry Reese has done on [MarcEdit](http://marcedit.reeset.net/). This program is not a replacement for MarcEdit.

The code is written in Go. I've found Go a very interesting and powerful programming language. Go's standard library provides most of the functionality that I need to parse MARC files. Go also supports creating binaries that can be deployed to Mac/Linux/Windows as single executable files which I love because I can deploy my executable to our Linux servers without having to do a complicated installation (i.e. no JVM needed, or Ruby bundle).


Expand Down Expand Up @@ -41,21 +39,26 @@ You can also pass `start` and `count` parameters to output only a range of MARC
## Sample data
Files under `./data/` are small MARC files that I use for testing.

* test_10.mrc has 10 MARC records
* test_1a.mrc is the first record of test_10.mrc
* test_1b.mrc is the second record of test_10.mrc
* test_10.mrc has 10 MARC records (MARC binary)
* test_1a.mrc is the first record of test_10.mrc (MARC binary)
* test_1b.mrc is the second record of test_10.mrc (MARC binary)
* small.xml has 10 MARC record (MARC XML)


## Getting started with the code
Download the code and play with it:

```
git clone https://github.com/hectorcorrea/marcli.git
cd marcli
cd marcli/cmd/marcli
go build
./marcli -file data/test_1a.mrc
```

## Code Structure

* `cmd/marcli` contains the code for the command line interface.
* `pkg/marc` contains the code to parse MARC files.

## Getting started (without the source code)
If you don't care about the source code, you can download the binary file appropriated for your operating system from the [releases tab](https://github.com/hectorcorrea/marcli/releases) and run it.
Expand Down
File renamed without changes.
6 changes: 3 additions & 3 deletions export/json.go → cmd/marcli/json.go
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
package export
package main

import (
"encoding/json"
"errors"
"fmt"
"hectorcorrea/marcli/pkg/marc"
"io"
"marcli/marc"
"os"
)

// TODO: Add support for JSONL (JSON line delimited) format that makes JSON
// easier to parse with Unix tools like grep, tail, and so on.
func ToJson(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
func toJson(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
if len(filters.Fields) > 0 {
return errors.New("filters not supported for this format")
}
Expand Down
11 changes: 5 additions & 6 deletions main.go → cmd/marcli/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ import (
"errors"
"flag"
"fmt"
"marcli/export"
"marcli/marc"
"hectorcorrea/marcli/pkg/marc"
"strings"
)

Expand Down Expand Up @@ -33,13 +32,13 @@ func main() {
searchValue := strings.ToLower(search)
filters := marc.NewFieldFilters(fields)
if format == "mrc" {
err = export.ToMrc(fileName, searchValue, filters, start, count)
err = toMrc(fileName, searchValue, filters, start, count)
} else if format == "mrk" {
err = export.ToMrk(fileName, searchValue, filters, start, count)
err = toMrk(fileName, searchValue, filters, start, count)
} else if format == "json" {
err = export.ToJson(fileName, searchValue, filters, start, count)
err = toJson(fileName, searchValue, filters, start, count)
} else if format == "solr" {
err = export.ToSolr(fileName, searchValue, filters, start, count)
err = toSolr(fileName, searchValue, filters, start, count)
} else {
err = errors.New("Invalid format")
}
Expand Down
6 changes: 3 additions & 3 deletions export/mrc.go → cmd/marcli/mrc.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
package export
package main

import (
"errors"
"fmt"
"hectorcorrea/marcli/pkg/marc"
"io"
"marcli/marc"
"os"
)

func ToMrc(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
func toMrc(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
if len(filters.Fields) > 0 {
return errors.New("filters not supported for this format")
}
Expand Down
6 changes: 3 additions & 3 deletions export/mrk.go → cmd/marcli/mrk.go
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
package export
package main

import (
"fmt"
"hectorcorrea/marcli/pkg/marc"
"io"
"marcli/marc"
"os"
)

func ToMrk(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
func toMrk(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
if count == 0 {
return nil
}
Expand Down
6 changes: 3 additions & 3 deletions export/solr.go → cmd/marcli/solr.go
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package export
package main

import (
"encoding/json"
"errors"
"fmt"
"hectorcorrea/marcli/pkg/marc"
"io"
"marcli/marc"
"os"
"strings"
)
Expand Down Expand Up @@ -62,7 +62,7 @@ func NewSolrDocument(r marc.Record) SolrDocument {
return doc
}

func ToSolr(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
func toSolr(filename string, searchValue string, filters marc.FieldFilters, start int, count int) error {
if len(filters.Fields) > 0 {
return errors.New("filters not supported for this format")
}
Expand Down
11 changes: 11 additions & 0 deletions data/small.xml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module marcli
module hectorcorrea/marcli

go 1.14
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit 8814f03

Please sign in to comment.