Description

ssteinhauser

opened on Jul 17, 2024

Issue body actions

The execution of the updateKeywords task adds the following tokens to RelObjectNameWithoutValue:

JSqlParser/src/main/jjtree/net/sf/jsqlparser/parser/JSqlParserCC.jjt

Lines 529 to 551 in 411a3da

	| <#TYPE_BIT: "BISTRING">
	| <#TYPE_BLOB: "BLOB" | "BYTEA" | <K_BINARY> | "VARBINARY" | <K_BYTES> >
	| <#TYPE_BOOLEAN: "BOOLEAN" | "BOOL" >
	| <#TYPE_ENUM: "ENUM" >
	| <#TYPE_MAP: "MAP" >
	| <#TYPE_DECIMAL: "DECIMAL" | "NUMBER" | "NUMERIC" >
	| <#TYPE_TINYINT: "TINYINT" | "INT1" >
	| <#TYPE_SMALLINT: "SMALLINT" | "INT2" | "SHORT" >
	| <#TYPE_INTEGER: ( "INTEGER" | "INT" | "INT4" | <K_SIGNED> | <K_UNSIGNED> ) >
	| <#TYPE_BIGINT: "BIGINT" | "INT8" | "LONG" >
	| <#TYPE_HUGEINT: "HUGEINT" >
	| <#TYPE_UTINYINT: "UTINYINT" >
	| <#TYPE_USMALLINT: "USMALLINT" >
	| <#TYPE_UINTEGER: "UINTEGER" >
	| <#TYPE_UBIGINT: "UBIGINT" >
	| <#TYPE_UHUGEINT: "UHUGEINT" >
	| <#TYPE_REAL: "REAL" | "FLOAT4" | "FLOAT">
	| <#TYPE_DOUBLE: "DOUBLE" | "PRECISION" | "FLOAT8" | "FLOAT64">
	| <#TYPE_VARCHAR: "NVARCHAR" | "VARCHAR" | "NCHAR" | <K_CHAR> | "BPCHAR" | "STRING" | "TEXT" | <K_CHARACTER> | "VARYING">
	| <#TYPE_TIME: "TIMETZ" >
	| <#TYPE_TIMESTAMP: "TIMESTAMP_NS" | "TIMESTAMP_MS" | "TIMESTAMP_S" >
	| <#TYPE_UUID: "UUID">

This causes the javacc-jjtree goal to fail:

Error: Line 1962, Column 313: String token "BISTRING" has been defined as a private regular expression.
Error: Line 1962, Column 1163: String token "ENUM" has been defined as a private regular expression.
Error: Line 1962, Column 1500: String token "HUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 1815: String token "MAP" has been defined as a private regular expression.
Error: Line 1962, Column 3401: String token "TIMETZ" has been defined as a private regular expression.
Error: Line 1962, Column 3527: String token "UBIGINT" has been defined as a private regular expression.
Error: Line 1962, Column 3542: String token "UHUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 3558: String token "UINTEGER" has been defined as a private regular expression.
Error: Line 1962, Column 3672: String token "USMALLINT" has been defined as a private regular expression.
Error: Line 1962, Column 3689: String token "UTINYINT" has been defined as a private regular expression.
Error: Line 1962, Column 3705: String token "UUID" has been defined as a private regular expression.

Activity

1. manticore-projects

   self-assigned this

   on Jul 18, 2024
2. manticore-projects

   added

   bug

   on Jul 18, 2024
3. manticore-projects commented on Jul 18, 2024

   

   manticore-projects

   on Jul 18, 2024

   Contributor

   More actions

   Thank you for reporting, I actually broke it myself when introducing complex compound Data Type tokens.
   I will definitely fix it, but this will take a couple of days since I am in transit right now. Sorry for any inconvenience.
4. ssteinhauser

   added a commit that references this issue on Jul 19, 2024

   fix: Avoid private tokens to be white listed

   This issue will close once commit c9feb23 is merged into the 'master' branch.

   c9feb23
5. ssteinhauser commented on Jul 19, 2024

   

   ssteinhauser

   on Jul 19, 2024

   ContributorAuthor

   More actions

   Thank you for the quick reply.
   I tried to implement and provide a solution. The idea is to parse and iterate over each token definition and don't whitelist anything which is defined within a private token.
   Let me know your thoughts on this: #2044

   However, there is still an issue, related to #2041. After running the updateKeywords now, PRECISION and BASE64 will be removed. PRECISION is removed, because it is just part of a private token. BASE64 is removed, because numbers in the token values are not being whitelisted due to the condition tokenValue.matches("[A-Za-z]+").
6. manticore-projects

   closed this as completedin #2044on Jul 20, 2024
7. manticore-projects

   closed this as completedin 821e92eon Jul 20, 2024
8. ssteinhauser

   added a commit that references this issue on Jul 20, 2024

   Avoid private tokens to be white listed and allow any word character …

   This issue will close once commit 821e92e is merged into the 'master' branch.

   821e92e

Issue actions