1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
|
2000-10-26 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_cache_open): Use IBEX_VERSION rather than
hardcoded version string.
* ibex_internal.h (IBEX_VERSION): Bumped version again. This time
I did change the index format.
* hash.c (struct _hashroot): Add a linked list of keys to the table.
(struct _hashblock): Added a next pointer as a block number.
(hash_insert): Link new key blocks into the key block list.
(struct _HASHCursor): Renamed block to key and added a block item.
(hash_cursor_next): Changed to go through the linked list of all
hash items rather than through each hash chain separately. >>
faster.
(ibex_hash_dump_rec): Remove a warning.
2000-10-25 <jpr@helixcode.com>
* ibex_block.c: No longer include <db.h>
2000-10-25 Not Zed <NotZed@HelixCode.com>
* ibex_internal.h (IBEX_VERSION): Bumped to another version. The
file format hasn't changed, but earlier bugs may create invalid
files.
* block.c (ibex_block_read): Use the root data directly.
(ibex_block_cache_open): As well.
(ibex_block_get): And here too.
(ibex_block_cache_sync): Sync the root block directly here.
* block.h: Pad root block out to 1024 bytes.
Added root block to struct _memcache.
* disktail.c (tail_get): Dirty the root block.
(tail_get): Fix for changes to root access.
(disk_remove): And here too.
* wordindexmem.c (sync_cache_entry): Handle the case of not having
any files in the list, which can happen now.
(word_index_pre): Make sure we set the wordid on the new cache
entry.
* ibex_block.c (ibex_save): Sigh. Pass the right argument to
index_post.
2000-10-24 JP Rosevear <jpr@helixcode.com>
* .cvsignore: Shush
2000-10-24 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_cache_open): Create a word_index_mem for
indexing the words, rather than a word_index.
* ibex_block.c (ibex_index_buffer): If we haven't called index_pre
yet, do it before indexing anything.
(ibex_save): If wehave called index_pre previously, call
index_post.
(ibex_close): And same for here.
* index.h: Added a cursor class, and cursor retrieval function for
iterating through an index's keys.
* wordindexmem.c (ibex_create_word_index_mem): New word class,
similar to wordindex, but meant to be faster for updates.
(word_index_pre): Implement. We load all keys into memory.
(word_index_post): Implement. We sync and free all keys.
(find): Remove lru code, its no longer a cache, but a lookup
table.
(add_index_cache): Remove lru code here too.
(find_name): And here.
(word_flush): Flush the hashtable direct.
(word_close): Call flush to flush, rather than doing it ourselves.
(add_index_cache): If we are in an index state, we can assume a
cache miss == a new word.
(word_index_post): Maintain whether or not we are in an index
state, and the depth of the state.
(word_index_pre): Likewise. Dont reread the index if we have
already.
(cache_sanity): Fixed for struct changes.
* wordindex.h (IBEXWordClass): Added functions to prepare/cleanup
for lots of indexing. i.e. can be used to optimise indexing speed
at the cost of extra memory usage during the indexing process.
* dumpindex.c: Dumps the contents of indexs.
* hash.c (ibex_hash_dump_rec): Also print the word count.
(hash_cursor_create): Create a new cursor for iterating through a
hashtable.
(hash_cursor_close): 'close' the cursor. It is upto the
application to close any cursors it creates.
(hash_cursor_next): Goto the next key id.
(hash_cursor_next_key): Goto the next key, reutrn the key.
(hash_get_cursor): Return a cursor object.
* wordindex.c (unindex_name): Cross-check the cache as well.
(word_index_post):
(word_index_pre): Added (empty) callbacks for pre/post functions.
2000-10-12 Not Zed <NotZed@HelixCode.com>
* ibex_internal.h (struct ibex): Bumped ibex rev.
* block.c (ibex_block_cache_open): Bumped the ibex file revision
because of the hash table size change.
* index.h: Added some stat stuff.
* wordindex.c (struct _wordcache): Changed files[] to be a pointer
to an allocated block/or an individual item.
(find): Fix for changes to struct.
(find_name): "
(sync_cache_entry): "
(add): "
(add_list): "
(add_index_cache): Free the cache file array if it was created.
(word_flush): And here.
(word_close): And here too.
(ibex_create_word_index): Double the size of the hashtables.
(word_flush): Make sure we reset the wordcount to 0 if we remove
the list items. DOH.
(add_index_cache): Use a slightly more sohpisticated aging
algorithm to remove expired nodes.
2000-10-10 Not Zed <NotZed@HelixCode.com>
* hash.c (hash_find):
(hash_remove):
(hash_insert): Truncate key if it is too big to fit in a
single block to MAX_KEYLEN bytes.
2000-09-28 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_free): Make sure we map the 'free' block to
a block number when unlinking a block (fixes a lot of assertion
failures).
(ibex_block_cache_open): Initialise sync flag on root block. If
it is not set on open then the index could be in an invalid state,
and should be rescanned.
(ibex_block_cache_sync): Sync root block last, and set the sync
flag.
(ibex_block_cache_open): Mirror root block flags in block_cache
struct.
(ibex_block_cache_sync): Likewise.
(ibex_block_read): If we write a dirty block, then we clear the
sync flag if its still set; we are no longer synced.
2000-09-19 Not Zed <NotZed@HelixCode.com>
** Merged from IBEX_DISK branch to head.
* file.c:
* find.c:
* words.c:
* index.c: Removed unused files.
* block.h: Changed block to use only 24 bits for next and 8 for
used, and fixed all relevant code. Some cleanup.
* disktail.c (tail_get): If we use an empty tail node, then make
sure we make it dirty.
2000-09-15 Not Zed <NotZed@HelixCode.com>
* wordindex.c (word_close): Free hashtable on exit too.
* disktail.c: Implemented tail-node storage for the end of long
lists, or for short lists. Should save significant disk space
(5x?).
Implemented special case for 1-item lists, where the tailnode
pointer is used to store the index entry.
2000-09-14 Not Zed <NotZed@HelixCode.com>
* wordindex.c (add_index_key): Keys also handle tails.
* hash.c (hash_set_data_block): Added new parameter to keys - a
tail block (a full 32 bit block pointer).
(hash_get_data_block): And same here.
2000-09-12 Not Zed <NotZed@HelixCode.com>
* wordindex.c (word_close): Dont close namestore twice.
2000-09-11 Not Zed <NotZed@HelixCode.com>
** Redid almost everything, on-disk hash table to store an index
to index records, mroe on the way to modularisation (more to go),
now stores reverse indexes for deleting.
2000-08-31 Not Zed <NotZed@HelixCode.com>
* block.c (add_key_mem): Initialise a memory based array for newly
added index entries.
(add_record): Changed to cache updates in memory until we hit a
limit, and then flush them to disk.
(get_record): Merge in-memory records with disk records.
(remove_record): Remove from memory first, and if that fails, goto
disk.
(find_record): Check memory first, then disk if that fails.
(add_datum_list): oops, copy size * sizeof(blockid_t)
(add_indexed): Make sure we link in the head node when we create a
new one.
2000-08-09 Christopher James Lahey <clahey@helixcode.com>
* file.c, find.c: Fixed some warnings.
2000-05-11 NotZed <NotZed@HelixCode.com>
* index.c (ibex_unindex): Make sure we mark the ibex as dirty.
2000-05-07 NotZed <NotZed@HelixCode.com>
* file.c (ibex_save): New function, only write out the ibex if it
has changed.
2000-05-07 <notzed@helixcode.com>
* file.c (ibex_open): Also close the fd after we're done.
* find.c (ibex_contains_name): New function to find out if a file
is indexed.
2000-05-02 Matt Loper <matt@helixcode.com>
* Makefile.am: set G_LOG_DOMAIN.
2000-04-12 NotZed <NotZed@HelixCode.com>
* find.c (ibex_dump_all): Debug function to dump the whole index
to stdout.
* words.c (get_ibex_file): Use g_strdup(), not strdup().
2000-04-11 NotZed <NotZed@HelixCode.com>
* file.c (write_word): Always write out all words we have (even if
its 0 ... the file expects it). No longer check for removed files.
(store_word): Check for removed files here, and only add to the
ordered tree if we have references left to this word.
(ibex_write): First insert into the tree, to determine the
wordcount to be saved in the output file, and then write that.
(ibex_open): Remove some debug.
* words.c (ibex_index_buffer): Always set 'unread', if it is a
valid pointer (dont rely on caller to initialise it).
2000-03-26 NotZed <NotZed@HelixCode.com>
* lookup.c (main): Fixed call to ibex_open.
* mkindex.c (main): Fixed call to ibex_open.
* file.c (ibex_open): Changed to accept flags and mode equivalent
to open(2).
2000-02-25 Dan Winship <danw@helixcode.com>
* *.c: add gtk-doc-style comments
2000-02-21 Matt Loper <matt@helixcode.com>
* .cvsignore: Added mkindex.
2000-02-21 NotZed <NotZed@HelixCode.com>
* Makefile.am: change noinst_LIBRARIES to noinst_LTLIBRARIES, and
supply -static to LDFLAGS. Duh, and changed LDADD back to
libibex.la.
2000-02-20 Matt Loper <matt@helixcode.com>
* Makefile.am: changed mkindex_LDADD to libibex.a instead of
libibex.la.
2000-02-19 Matt Loper <matt@helixcode.com>
* .cvsignore: added lookup.
2000-02-18 Miguel de Icaza <miguel@nuclecu.unam.mx>
* Makefile.am (lookup_LDADD): For now. make a libibex.a library so
we can link it with the camel provider. I hate libtool
2000-02-16 Dan Winship <danw@helixcode.com>
* Makefile.am: automakify
2000-02-16 NotZed <NotZed@HelixCode.com>
* find.[ch] (ibex_find_name): Finds if a word is indexed under a
given name.
2000-02-14 NotZed <notzed@zedzone.helixcode.com>
* Makefile: Hack together a build using libtool. This should all
be auto*'d at some point I guess.
2000-02-13 NotZed <notzed@zedzone.helixcode.com>
* Added ChangeLog file.
|
