Description
Many similar tests with same coverage are generated and not minimized.
Socket is created inside tests.
Fuzzing only mode is on.
To Reproduce
- Run a project in IntelliJ Idea 2022.1.3 - 2022.2.2
- Install RC plugin build
- Set Fuzzing to 100%
- Add the following class (originally posted in Add SecurityManager support to block suspicious code #622 #625 ):
public class SecurityCheck {
public int connect(Socket socket) throws IOException {
socket.connect(new InetSocketAddress("0.0.0.0", 22));
return 0;
}
}
- Use plugin to generate tests
- Open the generated test
Expected behavior
There should be only one test creating Socket and sandboxed due to an unpermitted operation.
Actual behavior
There are 272 tests generated by Fuzzing with similar arguments. They all have annotation
@Disabled(value = "Disabled due to sandbox")
///region FUZZER: EXPLICITLY_THROWN_UNCHECKED_EXCEPTIONS for method connect(java.net.Socket)
@Test
@DisplayName("connect: socket = Socket(String, int, boolean)")
@Disabled(value = "Disabled due to sandbox")
public void testConnect() throws IOException {
SecurityCheck securityCheck = new SecurityCheck();
Socket socket = new Socket("jkmqh", 568849899, true);
/* This test fails because method [org.utbot.examples.SecurityCheck.connect] produces [java.security.AccessControlException: access denied ("java.net.SocketPermission" "jkmqh" "resolve")] */
}
@Test
@DisplayName("connect: socket = Socket(String, int, boolean)")
@Disabled(value = "Disabled due to sandbox")
public void testConnect1() throws IOException {
SecurityCheck securityCheck = new SecurityCheck();
Socket socket = new Socket("qylov", -469477656, true);
/* This test fails because method [org.utbot.examples.SecurityCheck.connect] produces [java.security.AccessControlException: access denied ("java.net.SocketPermission" "qylov" "resolve")] */
}Visual proofs (screenshots, logs, images)
There are the following warnings in concrete executor logs:
Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor
to field java.net.Socket.created
Expand for details
19:03:52.187 | Info | DynamicClassTransformer | 16:Scheduler for Client56431 @coroutine#10 | Transforming: org/utbot/examples/SecurityCheck
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.created
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.bound
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.connected
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.closed
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.closeLock
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.shutIn
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.shutOut
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.impl
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.oldImpl
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.factory
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.options
WARNING: Illegal reflective access by org.utbot.framework.concrete.UtModelConstructor (file:/C:/Users//AppData/Roaming/JetBrains/IntelliJIdea2022.1/plugins/utbot-intellij/lib/utbot-framework-2022.10.417.jar) to field java.net.Socket.optionsSet
Environment
Windows 10 Pro
IntelliJ IDEA 2022.1.3, 2022.1.4, 2022.2.2
Additional context
Originally posted by @alisevych in #1125 (comment)
