Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: testing docker linter #911

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ env:

jobs:
########################
# Format, compileation and lint check
# Format, compilation, and lint check
########################
lint-check:
name: Format, compilation and lint check
Expand Down
16 changes: 12 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ PKG := github.com/btcsuite/btcwallet
LINT_PKG := github.com/golangci/golangci-lint/cmd/golangci-lint
GOACC_PKG := github.com/ory/go-acc
GOIMPORTS_PKG := golang.org/x/tools/cmd/goimports
TOOLS_DIR := tools

GOPATH := $(shell go env GOPATH)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: LINT_COMMIT below is now unused.
It would also be nice if we could remove the GOACC_COMMIT and GOIMPORTS_COMMIT values as well by creating a go.mod file in the tools directory that pins these versions, then just install them from the tools directory as we do here: https://github.com/lightninglabs/taproot-assets/blob/main/Makefile#L79

GO_BIN := ${GOPATH}/bin
LINT_BIN := $(GO_BIN)/golangci-lint
GOACC_BIN := $(GO_BIN)/go-acc

LINT_COMMIT := v1.46.0
Expand All @@ -31,7 +32,10 @@ ifneq ($(workers),)
LINT_WORKERS = --concurrency=$(workers)
endif

LINT = $(LINT_BIN) run -v $(LINT_WORKERS)
DOCKER_TOOLS = docker run \
-v $(shell bash -c "go env GOCACHE || (mkdir -p /tmp/go-cache; echo /tmp/go-cache)"):/tmp/build/.cache \
-v $(shell bash -c "go env GOMODCACHE || (mkdir -p /tmp/go-modcache; echo /tmp/go-modcache)"):/tmp/build/.modcache \
-v $$(pwd):/build btcwallet-tools

GREEN := "\\033[0;32m"
NC := "\\033[0m"
Expand Down Expand Up @@ -104,6 +108,10 @@ unit-race:
# UTILITIES
# =========

docker-tools:
@$(call print, "Building tools docker image.")
docker build -q -t btcwallet-tools $(TOOLS_DIR)

#? fmt: Fix imports and formatting source
fmt: goimports
@$(call print, "Fixing imports.")
Expand All @@ -112,9 +120,9 @@ fmt: goimports
gofmt -l -w -s $(GOFILES_NOVENDOR)

#? lint: Lint source
lint: $(LINT_BIN)
lint: docker-tools
@$(call print, "Linting source.")
$(LINT)
$(DOCKER_TOOLS) golangci-lint run -v $(LINT_WORKERS)

#? clean: Clean source
clean:
Expand Down
1 change: 1 addition & 0 deletions chain/interface.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ type Interface interface {
GetBlock(*chainhash.Hash) (*wire.MsgBlock, error)
GetBlockHash(int64) (*chainhash.Hash, error)
GetBlockHeader(*chainhash.Hash) (*wire.BlockHeader, error)
GetBlockHeight(*chainhash.Hash) (int32, error)
IsCurrent() bool
FilterBlocks(*FilterBlocksRequest) (*FilterBlocksResponse, error)
BlockStamp() (*waddrmgr.BlockStamp, error)
Expand Down
17 changes: 17 additions & 0 deletions tools/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM golang:1.19.13-bookworm

RUN apt-get update && apt-get install -y git
ENV GOCACHE=/tmp/build/.cache
ENV GOMODCACHE=/tmp/build/.modcache

COPY . /tmp/tools

RUN cd /tmp \
&& mkdir -p /tmp/build/.cache \
&& mkdir -p /tmp/build/.modcache \
&& cd /tmp/tools \
&& go install -trimpath github.com/golangci/golangci-lint/cmd/[email protected] \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here, re pinning the version in a go.mod file instead.

&& chmod -R 777 /tmp/build/ \
&& git config --global --add safe.directory /build
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We added this line when we didn't know what the actual issue was and it's more of an ugly hack. The "better" solution would be this environment variable: https://github.com/lightninglabs/pool/blob/master/tools/Dockerfile#L6

So just ENV GOFLAGS="-buildvcs=false".

We should probably update this in other projects too to make things more consistent (and to avoid us copying the "hacky" version to new projects).


WORKDIR /build
53 changes: 53 additions & 0 deletions tools/check-go-version-dockerfile.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/bin/bash

# Function to check if the Dockerfile contains only the specified Go version
check_go_version() {
local dockerfile="$1"
local required_go_version="$2"

# Use grep to find lines with 'FROM golang:'
local go_lines=$(grep -i '^FROM golang:' "$dockerfile")

# Check if all lines have the required Go version
if echo "$go_lines" | grep -q -v "$required_go_version"; then
echo "Error: $dockerfile does not use Go version $required_go_version exclusively."
exit 1
else
echo "$dockerfile is using Go version $required_go_version."
fi
}

# Check if the target Go version argument is provided
if [ $# -eq 0 ]; then
echo "Usage: $0 <target_go_version>"
exit 1
fi

target_go_version="$1"

# We find target files using the 'find' command in conjunction with the 'read'
# command. We exclude some directories from the search.
#
# We use the 'read' command to help ensure that we correctly handle filenames
# with spaces, newlines, and special characters. The '-print0' option in 'find'
# outputs filenames separated by a null character. This allows the 'read'
# command in the while loop to accurately distinguish each filename. The
# 'target_files' array is then populated, preserving the integrity of each
# filename. This approach ensures safe handling of filenames, regardless of
# their complexity.
while IFS= read -r -d '' file; do
target_files+=("$file")
done < <(find . \
-path ./vendor -prune -o \
-type f \
\( -name "*.Dockerfile" -o -name "Dockerfile" \) \
-print0 \
)

# Check for the expected Go version in each file.
for file in "${target_files[@]}"; do
check_go_version "$file" "$target_go_version"
done


echo "All Dockerfiles pass the Go version check for Go version $target_go_version."
63 changes: 63 additions & 0 deletions tools/check-go-version-yaml.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/bash

# Function to check if the YAML file contains the specified Go version after 'GO_VERSION:'
check_go_version() {
local yamlfile="$1"
local required_go_version="$2"

# Use grep to find lines with 'GO_VERSION:'
local go_lines=$(grep -i 'GO_VERSION:' "$yamlfile" || true) # Ignore grep exit status

# Check if any lines specify the Go version
if [ -n "$go_lines" ]; then
# Extract the Go version from the file's lines. Example matching strings:
# GO_VERSION: "1.21.0"
# GO_VERSION: '1.21.0'
# GO_VERSION: 1.21.0
# GO_VERSION:1.21.0
# GO_VERSION:1.21.0
local extracted_go_version=$(echo "$go_lines" | sed -n 's/^[[:space:]]*GO_VERSION:[[:space:]]*\(['\''"]\?\)\?\([0-9]\+\.[0-9]\+\.[0-9]\+\)\(['\''"]\?\)\?/\2/p')

# Check if the extracted Go version matches the required version
if [ "$extracted_go_version" != "$required_go_version" ]; then
echo "Error: $yamlfile specifies Go version '$extracted_go_version', but not version '$required_go_version'."
exit 1
else
echo "$yamlfile specifies Go version $required_go_version."
fi
fi
}

# Check if the target Go version argument is provided
if [ $# -eq 0 ]; then
echo "Usage: $0 <target_go_version>"
exit 1
fi

target_go_version="$1"

# We find target files using the 'find' command in conjunction with the 'read'
# command. We exclude some directories from the search.
#
# We use the 'read' command to help ensure that we correctly handle filenames
# with spaces, newlines, and special characters. The '-print0' option in 'find'
# outputs filenames separated by a null character. This allows the 'read'
# command in the while loop to accurately distinguish each filename. The
# 'target_files' array is then populated, preserving the integrity of each
# filename. This approach ensures safe handling of filenames, regardless of
# their complexity.
while IFS= read -r -d '' file; do
target_files+=("$file")
done < <(find . \
-path ./vendor -prune -o \
-type f \
\( -name "*.yaml" -o -name "*.yml" \) \
-print0 \
)

# Check for the expected Go version in each file.
for file in "${target_files[@]}"; do
check_go_version "$file" "$target_go_version"
done

echo "All YAML files pass the Go version check for Go version $target_go_version."
4 changes: 4 additions & 0 deletions wallet/mock.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ func (m *mockChainClient) GetBlockHeader(*chainhash.Hash) (*wire.BlockHeader,
return nil, nil
}

func (m *mockChainClient) GetBlockHeight(*chainhash.Hash) (int32, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same method needs to be added to the RPCClient.

return 0, nil
}

func (m *mockChainClient) IsCurrent() bool {
return false
}
Expand Down
Loading