Skip to content

Order by on an indexed property does not work anymore after sequence of insert -> delete -> insertΒ #1152

Closed

Description

@vic0824

ArcadeDB Version:

v23.6.1-SNAPSHOT (build 47be5db/1687764832131/main)

OS and JDK Version:

Windows 10 10.0 - OpenJDK 64-Bit Server VM 11.0.12 (Corretto-11.0.12.7.1)

Expected behavior

the sql clause "order by" produces the expected result in all scenarios, regardless of whether the property is indexed or not.

Actual behavior

If I insert some records in a type, delete them, insert them again, a select with order by on an indexed property returns 0 records.
The same query executed without order by returns the correct number of records.
The same query executed with an order by on a non-indexed property returns the correct number of records.

Steps to reproduce

Execute the attached test case.
This is the result I get:

2023-06-26 10:43:14.134 INFO  [ArcadeDBServer] <ArcadeDB_0> ArcadeDB Server v23.6.1-SNAPSHOT (build 47be5dba00ab1b6f2912689078e7deb09bb83733/1687764832131/main) is starting up...
2023-06-26 10:43:14.139 INFO  [ArcadeDBServer] <ArcadeDB_0> Running on Windows 10 10.0 - OpenJDK 64-Bit Server VM 11.0.12 (Corretto-11.0.12.7.1)
2023-06-26 10:43:14.142 INFO  [ArcadeDBServer] <ArcadeDB_0> Starting ArcadeDB Server in development mode with plugins [] ...
2023-06-26 10:43:14.332 INFO  [HttpServer] <ArcadeDB_0> - Starting HTTP Server (host=0.0.0.0 port=2480-2489 httpsPort=2490-2499)...
2023-06-26 10:43:14.750 INFO  [HttpServer] <ArcadeDB_0> - HTTP Server started (host=0.0.0.0 port=2480 httpsPort=2490)
2023-06-26 10:43:14.752 INFO  [ArcadeDBServer] <ArcadeDB_0> Available query languages: [sqlscript, java, js, sql]
2023-06-26 10:43:14.753 INFO  [ArcadeDBServer] <ArcadeDB_0> ArcadeDB Server started in 'development' mode (CPUs=8 MAXRAM=3.93GB)
2023-06-26 10:43:14.754 INFO  [ArcadeDBServer] <ArcadeDB_0> Studio web tool available at http://172.31.32.1:2480 

2023-06-26 10:43:15.046 INFO  [ArcadeDBServer] <ArcadeDB_0> Shutting down ArcadeDB Server...
2023-06-26 10:43:16.031 INFO  [ArcadeDBServer] <ArcadeDB_0> - Stop JMX Metrics
2023-06-26 10:43:16.031 INFO  [ArcadeDBServer] <ArcadeDB_0> ArcadeDB Server is downException in thread "main" org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31)
	at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:180)
	at eu.serco.csrs.persistence.ArcadeDBTest.testOrderByAfterDeleteInsert(ArcadeDBTest.java:460)
	at eu.serco.csrs.persistence.ArcadeDBTest.main(ArcadeDBTest.java:157)

testOrderByAfterDeleteInsert.zip

Activity

  1. vic0824 commented on Jun 30, 2023

    @vic0824
    Author

    I attach a snapshot of the database after the second insert, i.e. when the assertion fails.

    Let me know if you need anything else to investigate.

    At the moment, this is the only thing that stops my migration from Hsqldb to ArcadeDB.

    databases.zip

  2. vic0824 commented on Jun 30, 2023

    @vic0824
    Author

    I have pinpointed the exact issue: order by does not work, but order by desc works.
    I have used the following code:

    final Collection<TypeIndex> indexes = database.getSchema().getType("Order").getIndexesByProperties("id");
    final Index index = indexes.iterator().next();
    System.out.println("index name = " + index.getName() + ", properties = " + index.getPropertyNames());
    IndexCursor iterator = ((RangeIndex) index).iterator(true);
    System.out.println("iterating ASC: ");
    while (iterator.hasNext()) {
        System.out.println(iterator.next().asDocument(true).toJSON());
    }
    System.out.println("iterating DESC: ");
    iterator = ((RangeIndex) index).iterator(false);
    while (iterator.hasNext()) {
        System.out.println(iterator.next().asDocument(true).toJSON());
    }
    

    which produces the following output:

    index name = Order[status,id], properties = [status, id]
    interating ASC: 
    interating DESC: 
    {"id":4,"processor":"SIR1LRM-7.1","trigger":"#19:0","vstart":1685447870057592000,"vstop":1685447930057592000,"status":"PENDING","@cat":"d","@type":"Order","@rid":"#2:3"}
    {"id":3,"processor":"SIR1LRM-7.1","trigger":"#19:0","vstart":1685447810057592000,"vstop":1685447870057592000,"status":"PENDING","@cat":"d","@type":"Order","@rid":"#2:2"}
    
  3. lvca commented on Jun 30, 2023

    @lvca
    Member

    Thanks, @vic0824 I was able to reproduce the issue, thanks to your test case, download and run πŸ‘

    The issue is in the way the cursors are initiated and it wasn't working correctly when the 1st item found was a deleted item.

    I've added this test case to the test suite in ConsoleAsyncInsertTest.testOrderByAfterDeleteInsert()

  4. self-assigned this
    on Jun 30, 2023
  5. added this to the 23.6.1 milestone on Jun 30, 2023
  6. vic0824 commented on Jul 1, 2023

    @vic0824
    Author

    Excellent! thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions