Data structure to handle the requirement of following use case

I am assuming you keys are at most d digits long (in decimal).

How about a normal hashtable and an additional 10*d two dimensional array (let's call it A) of sets. A[i][j] is the set of keys which have digit i in the jth position. The sets can support O(1) insert/delete if implemented themselves as hashtables.

For 1 and 2, I think Linked Hash Map is a good choice.

For the point 3, an additional Hash map with (digit, position) tuple as key and list of pointers to the values.

Both data structures can be wrapped inside one, and both will point to the same data, of course.

Store the keys in a trie. For the numbers in your example (assuming 4 digit numbers) it looks like this:

*root*
 |
 0 -- 2 - 5 - 6
 | |
 | +- 3 - 6 - 2
 | |
 | +- 8 - 7 - 4
 |
 1 - 4 - 5 - 2

This data structure can be traversed in a way that returns (1) or (3). It won't be quite as fast for (3) as would maintaining an index for each digit, so I guess it's a question of whether space or lookup time is your primary concern. For (2), it is already O(log n), but if you need O(1), you could store the keys in both the trie and a hash table.

The first thing that comes to mind is embedding a pair of nodes in each record. One of the nodes would be a part of a tree sorted by the record index and the other, part of a tree sorted by the record key. You can then have quick access to the records by index or key using these trees. With this you can also quickly visit records in sequential index or key order. This covers the first and second requirement.

You can add another node for a tree of records whose values contain 5 in the tens position. That covers the third requirement.

Extra benefit: the same tree handling code will be used in all cases.

A dictionary (hash map, etc.) would easily handle those requirements, although your third requirement would be an O(N) operation. You just iterate over the keys and select those that match your criteria. You don't say what your desired performance is for that case.

But O(N) might be plenty fast enough. How many items are in your data set, and how often will you be performing that third function?