Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SQL AST comparison metric #62

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

devin-ai-integration[bot]
Copy link

Pull Request Description

Summary

This pull request introduces a new SQL AST comparison metric to the continuous-eval repository. The new metric, SQLASTSimilarity, compares SQL queries using Abstract Syntax Tree (AST) similarity, leveraging the sqlglot library.

Changes

  • Added the SQLASTSimilarity class to the code_deterministic_metrics.py file.
  • Imported the diff and parse_one functions from the sqlglot library.
  • Imported the Keep class from the sqlglot.diff module.
  • Implemented the __call__ method in the SQLASTSimilarity class to parse SQL queries into ASTs and calculate similarity scores.
  • Implemented the _calculate_similarity method in the SQLASTSimilarity class to calculate the similarity score between two ASTs by using the diff function to get the differences between the trees, counting the total changes, and calculating the total number of nodes in both trees. The similarity score is calculated as 1 - (total_changes / total_nodes).

Testing

  • Created a new test file, test_code_deterministic_metrics.py, with unit tests for the SQLASTSimilarity class.
  • Added test methods to validate the functionality of the SQLASTSimilarity class, including tests for exact match, different queries, similar queries, and invalid queries.
  • Ran the tests using pytest, and all tests passed successfully.

Link to Devin run

https://preview.devin.ai/devin/696032ba45654233968d6a04f2bc5df3

Request for Review

Please review the changes and provide feedback. If everything looks good, kindly approve the pull request for merging.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
0 participants