Vitess v2.0.0-rc.2

Changes since v2.0.0-rc.1

Bug Fixes

• Automatically create /zk root if it doesn't exist
• Fix edge cases in MariaDB backup/restore + reparent scenarios
• Randomize MySQL server-id when initializing tablet from scratch
  • Fixes edge case of replicate-same-server-id=0 together with backup/restore of former master.
• Fix for panic during resharding with unicode_loose_md5 vindex
• Fix for ApplySchema fails on DDLS which don't result in a schema change
• Fix for using SplitQuery with VTGate High-Level API
• VSchema for vtcombo is per-keyspace
  • This better matches the per-keyspace nature of VSchema in topology.
• Verify that autocommit is ON
  • This is required for proper operation of connection pooling.
• ApplySchema now reloads schema on slaves
  • Previously, only the master would immediately reload schema.
• Enable admin commands on vtcombo

Plugins

• Add backup storage plugin for Amazon S3

Vitess v2.0.0-rc.1

Changes since v2.0.0-beta.2

This is the first release candidate for v2.0.0. We are now freezing the release-2.0 branch for anything except bug fixes.

Client API

• Allow specifying a connection-wide default keyspace for VTGateV3 queries.
  (#1725,
  #1727,
  #1738)
  • Similar to the concept of a default database for a MySQL client connection.

SQL Support

• Preserve case of column names in query result field info.
• Preserve trailing comments in queries rewritten by VTGateV3.
• Support string-based hash vindexes
  • Previously, the value to hash had to be numeric or binary.

Management

• Hide VtTablet* and VtGate* commands behind a flag in vtctl/vtctld
  (#1700).
  • These commands are mostly meant for development, and should usually not be
    allowed in production.
  • Launch vtctld with the -enable_queries flag to allow these commands.
• Restart replication automatically
  if a tablet is restarted (e.g. as part of a Vitess upgrade).
• Optionally allow the use of insecure caller ID.
• It's no longer required to set a "sharding column name" when using VTGateV3.
  • i.e. no more vtctl SetKeyspaceShardingInfo when using VTGateV3.

Schema and VSchema

• VSchema config format changed
  • Some fields have been renamed for consistency.
• VSchema updates are now denormalized into per-cell records.
  (#1750)
  • No longer need to restart VTGate to pick up VSchema for new/removed
    keyspaces.
  • VSchema is still available if global topology goes down.
  • It's now possible to atomically move tables between keyspaces, for example
    when doing a vertical split.
  • NOTE: Even if you don't use VSchema, it's now required to run
    RebuildVSchemaGraph
    to initialize these records for a new keyspace.
• Schema changes are tested as a group, rather than individually.
  (#1749)
  • This allows batches of schema changes that depend on one another (#1731).

Interoperability

• Support MySQL 5.7
• Orchestrator Integration
  • Along with some patches contributed to Orchestrator, these new Vitess
    features allow the two systems to work together automatically.
    We'll be updating the documentation soon to show how to use it.

Code Cleanup

• Remove deprecated "serving graph" system, which had already been replaced by
  streaming health checks.
  • #1679,
    #1683,
    #1685,
    #1698,
    #1699,
    #1706
• Remove deprecated rowcache
  • This will be replaced by a caching scheme that makes sense for both VTGateV2
    and VTGateV3.
• Remove deprecated -target_tablet_type flag
  • Use -init_tablet_type instead.

Vitess v2.0.0-beta.2

Changes since v2.0.0-beta.1

Client Libraries

• JDBC-compatible wrapper
  • Contributed by Flipkart
  • See JDBC Example
• Update Java client to better match semantics of JDBC ResultSet
  • columnIndex now starts at 1
  • columnLabel is now case-insensitive
• Java: Synchronous client wrapper
  • To simplify and enforce proper usage of Java client in blocking mode.
• Java: Support more bind variable types
• Python: Always use colon style for bind variables
  • Removes extra translation layer for % formatting from Python client.
  • This makes the bind variable style consistent across all Vitess clients.
  • See local example client changes

SQL Support

• Phase 2 of VTGate V3 API
  • Can now route more complex queries.
  • Can perform cross-shard joins.
  • See phase 2 of V3 design for what is now supported.
• Cross-shard sequences
  • Note that NEXT is now a reserved word, and must be back-quoted if used as a column name.
• Support INTERVAL construct
• Support keyspace-qualified names
  • i.e. SELECT * FROM keyspace.table, analogous to db.table outside Vitess.
• Support STRAIGHT_JOIN and UNION DISTINCT SELECT

Resharding

• Support merging shards
• Support resharding tables with no keyspace_id column
  • As long as there is a VSchema that allows Vitess to compute the keyspace_id.
• SplitDiff: support custom sharding column name

Deployment and Management

• Configure SSL and authentication for gRPC
  • See Transport Security Model docs.
• VTGate watches for VSchema updates and refreshes automatically
• V3: Allow duplicate table names in vschema
  • As long as they're in different keyspaces.
• V3: Simplify VSchema
  • Remove the Class concept
  • VSchema is now per-keyspace
• Removed deprecated gorpc-based protocols
  • All protocols are now based on gRPC.
• Ceph Backup Storage Plugin
  • Contributed by Flipkart

Build Dependencies

• Pre-built Docker images for Percona Server
• We now depend on Go 1.5+ for native vendoring

Analysis

• Pre-scan algorithm for MapReduce/Hadoop query splitting

Vitess v2.0.0-beta.1

Changes since v2.0.0-alpha5

Client Libraries

• VTGateV3 Go driver
  can now also support certain types of VTGateV2 queries
• Publish PHP client to Packagist
• Refactor PHP client for better standards compliance
• PHP client returns associative array rows
• Java client is asynchronous by default
• Java client now returns 0 when converting SQL NULL to primitive types
• Python client now defaults to gRPC
• More sample clients for examples/local
  • In addition to Python,
    there are now sample clients for Go,
    PHP,
    and Java.

Query Service

• query.proto revamp
  • Fix NULL handling in new proto3-based query result format.
  • Unify type system for bind variables and query results.
• query_rules improvements
  • Improved JSON syntax for bindvar constraints.
  • Debug URL for viewing rule sets.
• Support REGEXP and RLIKE in
  SQL parser

Build Dependencies

• Default ZooKeeper version upgraded
  from 3.3.5 to 3.4.6.
• Update to gRPC 0.13
• Support building on OS X

Deployment and Management

• Add vtctld server to vtcombo
• Kubernetes config updates
  • Use Kubernetes DNS for service discovery
    and etcd bootstrap.
  • Kubernetes example for MySQL 5.6.
  • Use Kubernetes proxy to link from vtctld to vttablet status pages.
• Make MySQL users and privileges configurable
  when bootstrapping MySQL data directory.
  • Uses mysql_install_db and init_db.sql
    instead of extracting the old mysql-db-dir.tbz archive.
• Support semi-sync without async fallback
• Remove old vtctld UI files
  • Deployments no longer need to include go/cmd/vtctld/templates folder or
    set -templates flag on vtctld.
• Add option to bypass sanity check in ApplySchema
• Print human-readable tables for CLI queries
  • e.g. for ExecuteFetchAsDba

Vitess v2.0.0-alpha5

Changes since v2.0.0-alpha4

• New built-in backup
  plugin to store backups in Google Cloud Storage.
  • See the Vitess on Kubernetes guide for an
    example of usage.
• The Vitess PHP client now
  uses gRPC.
• Vitess Hadoop connector
  now uses the new gRPC-based Java client.
• Self-contained demo
  for VTGate V3, which is a preview feature as
  of Vitess 2.0.
• The vtctlclient-to-vtctld protocol now defaults to gRPC.

Vitess v2.0.0-alpha4

Changes since v2.0.0-alpha3

• Fixes for "bad node version" error when using etcd. (#1176, #1184)
• Java client is now feature-complete.
  • The last piece was our new canonical error code translation.

Vitess v2.0.0-alpha3

Changes since v2.0.0-alpha2

• Better PEP 0249 compliance in Python client.
• Type-checking result cursor for Java client.
• VTGate automatically adds annotations for filtered replication.
• All RPC protocols are now off by default, and must be explicitly enabled with the -service_map flag.
  • Previously the BSON protocols were enabled by default, and had to be explicitly disabled in -service_map if unwanted.
• SQL is now consistently represented as a UTF-8 string.
  • Previously it was represented in some places as a byte array.
• Update to gRPC beta.

Vitess v2.0.0-alpha2

Changes since v2.0.0-alpha1

• Fix for breaking API change in grpc-go (#1052).

Vitess v2.0.0-alpha1

This is the first alpha cut for the 2.0.0 release.

Changes since 1.0

• In addition to the manual deployment
  option, Vitess can now be scaled dynamically in any cloud platform supported
  by Kubernetes, including
  Google Container Engine and
  AWS.
  • Getting Started on Kubernetes
  • Dynamic Resharding in Kubernetes
• Docker images available for both manual and cloud deployments:
  • Automated Builds
  • Lightweight Builds
• Official client libraries for:
  • Java
  • Python
  • PHP
  • Go
  • NOTE: Error handling is still being refactored as of alpha1.
• Using new gRPC framework built on HTTP/2.
  • For Vitess server-to-server communication.
  • For Java and Go client communication.
• Underlying DB nodes can run either:
  • MariaDB 10.0, or
  • MySQL 5.6
• Consistent datastore used for cluster management can be:
  • etcd 2.0
  • ZooKeeper
  • Pluggable interface available for adding more.
• New responsive admin dashboard (on vtctld).
  • Client-side AngularJS app with Material Design.
• Built-in Backup/Restore
  for disaster recovery and provisioning.
  • New replica instances automatically restore from latest backup, and then
    catch up to master.
  • Queries from the app are automatically routed only to healthy, caught-up
    replicas.
  • Restore mechanism is routinely exercised as part of normal provisioning,
    so you know it'll work when you really need it.
  • NOTE: Currently this requires a network mount. We plan to add plugins for
    direct access to blob stores like Google Cloud Storage and Amazon S3.
• Streaming, filtered data copy for transparent dynamic resharding.
  • Resharding now requires no additional disk space on the source machine,
    so it works even when existing data fills up more than half of the largest
    available node size.
• GTID-based MySQL failovers (also known as reparenting)
  allow old masters to be seamlessly reparented back into the replication stream
  as replicas.
  • This makes failover a routine, automated operation, allowing downtime-free
    workflows such as:
    • Reparent to another replica.
    • Upgrade the old master by restarting it.
    • Reparent to upgraded master.
    • Upgrade remaining replicas one at a time.