Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Refactor the validation logic in the handle methods #5861

Open
Abyss-lord opened this issue Dec 15, 2024 · 18 comments · May be fixed by #5972
Open

[Improvement] Refactor the validation logic in the handle methods #5861

Abyss-lord opened this issue Dec 15, 2024 · 18 comments · May be fixed by #5972
Assignees
Labels
improvement Improvements on everything

Comments

@Abyss-lord
Copy link
Contributor

What would you like to be improved?

Current Issues

  1. The executeCommand method contains a large number of if-else structures, which leads to poor readability and scalability.
  2. The specific entity execution commands (e.g., handleXXXCommand) suffer from the same problem.
  3. The parameter validation logic should placed in the handleXXXCommand layer.

How should we improve?

Resolve the issue of excessive if-else statements in the executeCommand method.

Start by refactoring the handleRoleCommand method to improve the CLI code. To resolve the issue of excessive if-else in the executeCommand method, refer to this pull request #5688 . We can create a Map to store the execution method corresponding to each entity. e.g.

private final Map<String, Consumer<Void>> entityMap = Maps.newHashMap();

private void initializeEntityMap() {
    entityMap.put(CommandEntities.COLUMN, v -> handleColumnCommand());
    entityMap.put(CommandEntities.TABLE, v -> handleTableCommand());
    entityMap.put(CommandEntities.SCHEMA, v -> handleSchemaCommand());
    entityMap.put(CommandEntities.CATALOG, v -> handleCatalogCommand());
    entityMap.put(CommandEntities.METALAKE, v -> handleMetalakeCommand());
    entityMap.put(CommandEntities.TOPIC, v -> handleTopicCommand());
    entityMap.put(CommandEntities.FILESET, v -> handleFilesetCommand());
    entityMap.put(CommandEntities.USER, v -> handleUserCommand());
    entityMap.put(CommandEntities.GROUP, v -> handleGroupCommand());
    entityMap.put(CommandEntities.TAG, v -> handleTagCommand());
    entityMap.put(CommandEntities.ROLE, v -> handleRoleCommand());
}

Resolve the issue of excessive if-else statements in parameter validation and specific command execution.

Similarly, construct a Map to store the execution method corresponding to each Command, for example:

private final Map<String, Consumer<Role>> roleCommandMap = Maps.newHashMap();
private void initializeRoleCommandMap() {
    roleCommandMap.put(CommandActions.DETAILS, v -> handleRoleDetailCommand());
    roleCommandMap.put(CommandActions.LIST, v -> handleListRoleCommand());
    roleCommandMap.put(CommandActions.CREATE, v -> handleRoleCreateCommand());
    roleCommandMap.put(CommandActions.DELETE, v -> handleRoleDeleteCommand());
}

In the original processing logic, the steps can be simplified as follows:

  1. Retrieve the necessary arguments.
  2. Determine the method to be executed.
  3. Pass the retrieved arguments to the corresponding method.
protected void handleRoleCommand() {
    String url = getUrl();
    String auth = getAuth();
    String userName = line.getOptionValue(GravitinoOptions.LOGIN);
    FullName name = new FullName(line);
    String metalake = name.getMetalakeName();
    String role = line.getOptionValue(GravitinoOptions.ROLE);

    Command.setAuthenticationMode(auth, userName);

    if (CommandActions.DETAILS.equals(command)) {
        newRoleDetails(url, ignore, metalake, role).handle();
    } else if (CommandActions.LIST.equals(command)) {
        newListRoles(url, ignore, metalake).handle();
    } else if (CommandActions.CREATE.equals(command)) {
        newCreateRole(url, ignore, metalake, role).handle();
    } else if (CommandActions.DELETE.equals(command)) {
        boolean force = line.hasOption(GravitinoOptions.FORCE);
        newDeleteRole(url, ignore, force, metalake, role).handle();
    }
}

One entity processing method will only call one command handling method. Therefore, is it possible to create a data class Role with two purposes:

  1. Validate whether the parameters are complete based on the command to be executed.
  2. Provide necessary prompt messages to inform the user of any missing parameters.

and in GravitinoCommandLine, use a variable to store the Role instance,current design as follows:
image

BaseEntity
The base class for all entities, designed to store common methods.

Role
Used for performing checks related to the Role entity.

Key attributes

  1. actionCheckMap: A dictionary that stores command validation methods.
  2. entityArgMap: Stores field names and values, for example, "METALAKE": metalake value.

key methods
public boolean checkArguments(String action): Checks whether the necessary arguments for the corresponding operation are defined.

Based on the design above, the related operations for the Role entity can be simplified to the following code:

/**
   * Create a role
   */
protected void handleRoleCreateCommand() {
    if (!roleDataObject.checkArguments(CommandActions.CREATE)) {
        return;
    }
    newCreateRole(
        roleDataObject.getUrl(), ignore, roleDataObject.getMetalake(), roleDataObject.getRole())
    .handle();
}

/**
   * Delete a role
   */
protected void handleRoleDeleteCommand() {
    if (!roleDataObject.checkArguments(CommandActions.DELETE)) {
        return;
    }
    boolean force = line.hasOption(GravitinoOptions.FORCE);
    newDeleteRole(
        roleDataObject.getUrl(),
        ignore,
        force,
        roleDataObject.getMetalake(),
        roleDataObject.getRole())
    .handle();
}

If a new command, such as remove, is supported for the role in the future, the expansion steps would be as follows:

  1. Override the checkRemoveArguments method from BaseEntity class to define the remove check logic
  2. Add a handleRoleRemoveCommand method in GravitinoCommandLine.
  3. Add handleRoleRemoveCommand to the roleCommandMap.
@Abyss-lord Abyss-lord added the improvement Improvements on everything label Dec 15, 2024
@Abyss-lord
Copy link
Contributor Author

Hi, @justinmclean @tengqm @xunliu @jerryshao , could you plz help to see if this design is reasonable when you have some time.

@justinmclean
Copy link
Member

I proposed a similar solution in PR #5688 @jerryshao thought that building command maps was too heavy. There is an outstanding PR to remove some of the if/else logic and use switch/cases instead. (PR #5793)

@tengqm
Copy link
Contributor

tengqm commented Dec 16, 2024

Yes. My favorite is the simple naive implementation using switch/case statements.
One key design consideration, in my view, is to make the code as straightforward as
possible. The key value of a software is not about its sophisticated class hierarchy design.
It is instead the business value for users and the elegant (simple) code base for developers.

@Abyss-lord
Copy link
Contributor Author

Abyss-lord commented Dec 16, 2024

I proposed a similar solution in PR #5688 @jerryshao thought that building command maps was too heavy. There is an outstanding PR to remove some of the if/else logic and use switch/cases instead. (PR #5793)

@justinmclean, @tengqm Can we combine both approaches(#5793 ) by using a switch/case structure for logical decisions, while also creating a Data class object to handle parameter validation and other logic operations?"

@Abyss-lord
Copy link
Contributor Author

@justinmclean @tengqm Another concern is the frequent issues with changing error messages. The root cause of this lies in parameter validation not being performed before executing specific operations. As a result, methods like FullName and NameIdentifier.of end up throwing exceptions. Personally, I would like to combine Justin's switch/case logic(#5793 ) with a unified approach to argument validation.

@tengqm
Copy link
Contributor

tengqm commented Dec 16, 2024

Personally, I would like to combine Justin's switch/case logic(#5793 ) with a unified approach to argument validation.

100% support from my side.

@xunliu
Copy link
Member

xunliu commented Dec 16, 2024

Personally, I would like to combine Justin's switch/case logic(#5793 ) with a unified approach to argument validation.

I think it ok, The big logic branch uses switch/case, and The detailed logic (parameter validation) uses Data class object.

@justinmclean What's do you thnk?

@justinmclean
Copy link
Member

My concern is that @jerryshao would prefer a different approach and it does add a lot of complexity, new methods and new objects. A better way, I think, would be to add a verify() method to commands and have that check arguments if needed. Note that only a couple of commands need to do this, not all of them. The CLI library does a lot of the work for us, so we don't need to duplicate what it does.

@Abyss-lord
Copy link
Contributor Author

My concern is that @jerryshao would prefer a different approach and it does add a lot of complexity, new methods and new objects. A better way, I think, would be to add a verify() method to commands and have that check arguments if needed. Note that only a couple of commands need to do this, not all of them. The CLI library does a lot of the work for us, so we don't need to duplicate what it does.

Hi @justinmclean , Can I draft a version first? I’d like to build on #5793 and submit a draft PR to show how it works.

@justinmclean
Copy link
Member

This is what is would look like. Add a verify method to the base Command class and a few common checkers.


  public Command verify() {
    return this;
  }

  public Command verifyTableName(String metalake, String catalog, String schema, String table) {
    if (metalake == null) {
      throw new IllegalArgumentException("Missing metalake name");
    }
    if (catalog == null) {
      throw new IllegalArgumentException("Missing catalog name");
    }
    if (schema == null) {
      throw new IllegalArgumentException("Missing schema name");
    }
    if (table == null) {
      throw new IllegalArgumentException("Missing table name");
    }
    return this;
  }

(we may not need to check for metalake)

in a command that needs to check table names add this:

  @Override
  public Command verify() {
    return verifyTableName(metalake, catalog, schema, table);
  }

When calling the command chain verify and handle together:

      newListColumns(url, ignore, metalake, catalog, schema, table).verify().handle();

@justinmclean
Copy link
Member

justinmclean commented Dec 16, 2024

Yep there is no need to check for metalake in the above code, as that is already done, and there is probably no need to check for catalog as name must have a value at this point, which means catalog is always set, so that method only needs schema and table passed to it to check.

@Abyss-lord
Copy link
Contributor Author

Yep there is no need to check for metalake in the above code, as that is already done, and there is probably no need to check for catalog as name must have a value at this point, which means catalog is always set, so that method only needs schema and table passed to it to check.

@justinmclean Should we throw an exception in the verify function? Or give details that are missing.

@justinmclean
Copy link
Member

justinmclean commented Dec 16, 2024

We would need to throw an exception so that no code in handle is called. The exception message can give the reason why.

@Abyss-lord
Copy link
Contributor Author

We would need to throw an exception so that no code in handle is called. The exception message can give the reason why.

hi, @justinmclean Adding the verify to Command I have a few concerns

  1. The scope of the change is too large, we need modify all subclass commands.
  2. The potential for excessive duplication in validation logic.
  3. The need to wrap the handle method in a try/catch block each time it is invoked.

@shaofengshi could you plz help to see?

@justinmclean
Copy link
Member

You do not have to modify all subclasses. Remember, the CLI library does a lot of the work for you before you get to this point, and there is no need to check for things that have already been checked. There should not be excessive duplication if you place common validation in methods in the Command base class. If the exception is an issue, you could get around that by using System.err.println and System.exit(-1)?

@Abyss-lord
Copy link
Contributor Author

Abyss-lord commented Dec 18, 2024

hi, @justinmclean Has the modification of the Command been completed? Specifically, has the verify method been added, or is someone else currently working on it? If not, could this task be assigned to me?

BTW, can this PR #5793 merged?

@justinmclean
Copy link
Member

It needs to be reviewed before it can be merged, I can't review or merge it as I wrote the code.

@Abyss-lord
Copy link
Contributor Author

It needs to be reviewed before it can be merged, I can't review or merge it as I wrote the code.

Got it. The modification of Command depends on #5793. This PR is highly valuable, and I hope it gets merged soon.

@Abyss-lord Abyss-lord changed the title [Improvement] Refactor the code for the role command. [Improvement] Refactor validation of handle methods Dec 24, 2024
@Abyss-lord Abyss-lord changed the title [Improvement] Refactor validation of handle methods [Improvement] Refactor the validation logic in the handle methods Dec 24, 2024
Abyss-lord added a commit to Abyss-lord/gravitino that referenced this issue Dec 24, 2024
…handle methods

refactor the validation logic of all entities and add test case.
Abyss-lord added a commit to Abyss-lord/gravitino that referenced this issue Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvements on everything
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants