Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

custom pointer to struct type scan issue #6

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

custom pointer to struct type scan issue #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
942711e
test case for pgx scan issue of *BigInt
pkieltyka Jul 23, 2021
4a454df
rebase, and update test
pkieltyka Jul 24, 2021
0045d1a
update test
pkieltyka Jul 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions dbtype/bigint.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ import (
"fmt"
"math/big"
"strings"

"github.com/jackc/pgtype"
"github.com/kr/pretty"
)

// BigInt is a type alias for big.Int used for JSON/Database marshalling.
Expand Down Expand Up @@ -175,3 +178,39 @@ func (b *BigInt) Scan(src interface{}) error {

return nil
}

// func (src *Point) AssignTo(dst interface{}) error {
// return fmt.Errorf("cannot assign %v to %T", src, dst)
// }

func (b BigInt) DecodeText(ci *pgtype.ConnInfo, src []byte) error {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can ignore this method, it was just me doing some hacking code. I was thinking that if pgx cannot support a null value of a normal database/sql scanner+valuer, then we can try the add binary encoding support here which might solve our issue in the end.

Alternatively, maybe there is another way to make pgx or scany work, and if not, finally we could adapt pgkit instead of using the pgx-direct driver, we update all of pgkit to use the database/sql interface which pgx also supports. I'm not sure exactly what we'd compromise by not using pgx-driver directly anymore, but it would be nice to know that too if we decide that..

pretty.Println(src)
// panic("geez")
err := b.Scan(src)
if err != nil {
panic(err)
}
return nil
}

func (dst *BigInt) Set(src interface{}) error {
panic("common")
// return fmt.Errorf("cannot convert %v to Point", src)
}

func (dst *BigInt) Get() interface{} {
panic("ahh")
// switch dst.Status {
// case pgtype.Present:
// return dst
// case pgtype.Null:
// return nil
// default:
// return dst.Status
// }
}

// BigInt pgx custom type assignment
func (src *BigInt) AssignTo(dst interface{}) error {
panic("wee")
}
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ require (
github.com/jackc/pgconn v1.9.0
github.com/jackc/pgtype v1.8.0
github.com/jackc/pgx/v4 v4.12.0
github.com/kr/pretty v0.1.0
github.com/stretchr/objx v0.3.0 // indirect
github.com/stretchr/testify v1.7.0
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97 // indirect
Expand Down
9 changes: 9 additions & 0 deletions pgkit.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,15 @@ func Connect(appName string, cfg Config) (*DB, error) {
}

func ConnectWithPGX(appName string, pgxConfig *pgxpool.Config) (*DB, error) {
// pgxConfig.AfterConnect = func(ctx context.Context, conn *pgx.Conn) error {
// conn.ConnInfo().RegisterDataType(pgtype.DataType{
// Value: &dbtype.BigInt{},
// Name: "numeric",
// OID: pgtype.NumericOID,
// })
// return nil
// }

pool, err := pgxpool.ConnectConfig(context.Background(), pgxConfig)
if err != nil {
return nil, fmt.Errorf("pgkit: failed to connect to db: %w", err)
Expand Down
31 changes: 31 additions & 0 deletions tests/pgkit_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/goware/pgkit"
"github.com/goware/pgkit/dbtype"
"github.com/jackc/pgx/v4"
"github.com/kr/pretty"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
Expand Down Expand Up @@ -331,6 +332,7 @@ func TestRowsWithBigInt(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, "count", sout.Key)
assert.True(t, sout.Num.Int64() == 2)
assert.Nil(t, sout.Rating)
}

// another one, big number this time
Expand All @@ -349,6 +351,35 @@ func TestRowsWithBigInt(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, "count2", sout.Key)
assert.True(t, sout.Num.String() == "12323942398472837489234")

pretty.Println(sout.Rating)

assert.Nil(t, sout.Rating)
}

// last, with opt rating
{
bv := dbtype.NewBigInt(5)

stat := &Stat{
Key: "count3",
Num: dbtype.NewBigIntFromString("44", 0),
Rating: &bv,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

passing *dbtype.BigInt here is causing an assignment issue to a nullable NUMERIC(78,0) type

}

// Insert
q1 := DB.SQL.InsertRecord(stat, "stats")
_, err := DB.Query.Exec(context.Background(), q1)
assert.NoError(t, err)

// Select
var sout Stat
q2 := DB.SQL.Select("*").From("stats").Where(sq.Eq{"key": "count3"})
err = DB.Query.GetOne(context.Background(), q2, &sout)
assert.NoError(t, err)
assert.Equal(t, "count3", sout.Key)
assert.True(t, sout.Num.String() == "44")
assert.True(t, sout.Rating.String() == "5")
}
}

Expand Down
7 changes: 4 additions & 3 deletions tests/schema_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,10 @@ type Log struct {
}

type Stat struct {
ID int64 `db:"id,omitempty"`
Key string `db:"key"`
Num dbtype.BigInt `db:"big_num"` // using NUMERIC(78,0) postgres datatype
ID int64 `db:"id,omitempty"`
Key string `db:"key"`
Num dbtype.BigInt `db:"big_num"` // using NUMERIC(78,0) postgres datatype
Rating *dbtype.BigInt `db:"rating"` // using NUMERIC(78,0) postgres datatype
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the problematic runtime type, of *dbtype.BigInt .. resulting in the error on rows.Scan(..) of:

=== RUN   TestRowsWithBigInt
(*dbtype.BigInt)(nil)
    pgkit_test.go:379: 
        	Error Trace:	pgkit_test.go:379
        	Error:      	Received unexpected error:
        	           	pgkit: scany: scan row into struct fields: can't scan into dest[3]: unable to assign to *dbtype.BigInt
        	Test:       	TestRowsWithBigInt
    pgkit_test.go:382: 
        	Error Trace:	pgkit_test.go:382
        	Error:      	Should be true
        	Test:       	TestRowsWithBigInt

}

type Article struct {
Expand Down
3 changes: 2 additions & 1 deletion tests/testdata/pgkit_test_db.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ CREATE TABLE logs (
CREATE TABLE stats (
id SERIAL PRIMARY KEY,
key VARCHAR(80),
big_num NUMERIC(78,0) -- representing a *big.Int runtime type
big_num NUMERIC(78,0) NOT NULL, -- representing a big.Int runtime type
rating NUMERIC(78,0) -- representing a *big.Int runtime type
);

CREATE TABLE articles (
Expand Down