Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
Summary
This pull request introduces a new SQL AST comparison metric to the
continuous-eval
repository. The new metric,SQLASTSimilarity
, compares SQL queries using Abstract Syntax Tree (AST) similarity, leveraging thesqlglot
library.Changes
SQLASTSimilarity
class to thecode_deterministic_metrics.py
file.diff
andparse_one
functions from thesqlglot
library.Keep
class from thesqlglot.diff
module.__call__
method in theSQLASTSimilarity
class to parse SQL queries into ASTs and calculate similarity scores._calculate_similarity
method in theSQLASTSimilarity
class to calculate the similarity score between two ASTs by using thediff
function to get the differences between the trees, counting the total changes, and calculating the total number of nodes in both trees. The similarity score is calculated as1 - (total_changes / total_nodes)
.Testing
test_code_deterministic_metrics.py
, with unit tests for theSQLASTSimilarity
class.SQLASTSimilarity
class, including tests for exact match, different queries, similar queries, and invalid queries.pytest
, and all tests passed successfully.Link to Devin run
https://preview.devin.ai/devin/696032ba45654233968d6a04f2bc5df3
Request for Review
Please review the changes and provide feedback. If everything looks good, kindly approve the pull request for merging.
Thank you!