KickJava   Java API By Example, From Geeks To Geeks.

Java > Open Source Codes > org > python > modules > cPickle


1 /*
2  * Copyright 1998 Finn Bock.
3  *
4  * This program contains material copyrighted by:
5  * Copyright (c) 1991-1995 by Stichting Mathematisch Centrum, Amsterdam,
6  * The Netherlands.
7  */

8
9 /* note about impl:
10   instanceof vs. CPython type(.) is .
11 */

12
13 package org.python.modules;
14
15 import java.util.*;
16
17 import org.python.core.*;
18 import org.python.core.imp;
19
20 /**
21  *
22  * From the python documentation:
23  * <p>
24  * The <tt>cPickle.java</tt> module implements a basic but powerful algorithm
25  * for ``pickling'' (a.k.a. serializing, marshalling or flattening) nearly
26  * arbitrary Python objects. This is the act of converting objects to a
27  * stream of bytes (and back: ``unpickling'').
28  * This is a more primitive notion than
29  * persistency -- although <tt>cPickle.java</tt> reads and writes file
30  * objects, it does not handle the issue of naming persistent objects, nor
31  * the (even more complicated) area of concurrent access to persistent
32  * objects. The <tt>cPickle.java</tt> module can transform a complex object
33  * into a byte stream and it can transform the byte stream into an object
34  * with the same internal structure. The most obvious thing to do with these
35  * byte streams is to write them onto a file, but it is also conceivable
36  * to send them across a network or store them in a database. The module
37  * <tt>shelve</tt> provides a simple interface to pickle and unpickle
38  * objects on ``dbm''-style database files.
39  * <P>
40  * <b>Note:</b> The <tt>cPickle.java</tt> have the same interface as the
41  * standard module <tt>pickle</tt>except that <tt>Pickler</tt> and
42  * <tt>Unpickler</tt> are factory functions, not classes (so they cannot be
43  * used as base classes for inheritance).
44  * This limitation is similar for the original cPickle.c version.
45  *
46  * <P>
47  * Unlike the built-in module <tt>marshal</tt>, <tt>cPickle.java</tt> handles
48  * the following correctly:
49  * <P>
50  *
51  * <UL><LI>recursive objects (objects containing references to themselves)
52  *
53  * <P>
54  *
55  * <LI>object sharing (references to the same object in different places)
56  *
57  * <P>
58  *
59  * <LI>user-defined classes and their instances
60  *
61  * <P>
62  *
63  * </UL>
64  *
65  * <P>
66  * The data format used by <tt>cPickle.java</tt> is Python-specific. This has
67  * the advantage that there are no restrictions imposed by external
68  * standards such as XDR (which can't represent pointer sharing); however
69  * it means that non-Python programs may not be able to reconstruct
70  * pickled Python objects.
71  *
72  * <P>
73  * By default, the <tt>cPickle.java</tt> data format uses a printable ASCII
74  * representation. This is slightly more voluminous than a binary
75  * representation. The big advantage of using printable ASCII (and of
76  * some other characteristics of <tt>cPickle.java</tt>'s representation) is
77  * that for debugging or recovery purposes it is possible for a human to read
78  * the pickled file with a standard text editor.
79  *
80  * <P>
81  * A binary format, which is slightly more efficient, can be chosen by
82  * specifying a nonzero (true) value for the <i>bin</i> argument to the
83  * <tt>Pickler</tt> constructor or the <tt>dump()</tt> and <tt>dumps()</tt>
84  * functions. The binary format is not the default because of backwards
85  * compatibility with the Python 1.4 pickle module. In a future version,
86  * the default may change to binary.
87  *
88  * <P>
89  * The <tt>cPickle.java</tt> module doesn't handle code objects.
90  * <P>
91  * For the benefit of persistency modules written using <tt>cPickle.java</tt>,
92  * it supports the notion of a reference to an object outside the pickled
93  * data stream. Such objects are referenced by a name, which is an
94  * arbitrary string of printable ASCII characters. The resolution of
95  * such names is not defined by the <tt>cPickle.java</tt> module -- the
96  * persistent object module will have to implement a method
97  * <tt>persistent_load()</tt>. To write references to persistent objects,
98  * the persistent module must define a method <tt>persistent_id()</tt> which
99  * returns either <tt>None</tt> or the persistent ID of the object.
100  *
101  * <P>
102  * There are some restrictions on the pickling of class instances.
103  *
104  * <P>
105  * First of all, the class must be defined at the top level in a module.
106  * Furthermore, all its instance variables must be picklable.
107  *
108  * <P>
109  *
110  * <P>
111  * When a pickled class instance is unpickled, its <tt>__init__()</tt> method
112  * is normally <i>not</i> invoked. <b>Note:</b> This is a deviation
113  * from previous versions of this module; the change was introduced in
114  * Python 1.5b2. The reason for the change is that in many cases it is
115  * desirable to have a constructor that requires arguments; it is a
116  * (minor) nuisance to have to provide a <tt>__getinitargs__()</tt> method.
117  *
118  * <P>
119  * If it is desirable that the <tt>__init__()</tt> method be called on
120  * unpickling, a class can define a method <tt>__getinitargs__()</tt>,
121  * which should return a <i>tuple</i> containing the arguments to be
122  * passed to the class constructor (<tt>__init__()</tt>). This method is
123  * called at pickle time; the tuple it returns is incorporated in the
124  * pickle for the instance.
125  * <P>
126  * Classes can further influence how their instances are pickled -- if the
127  * class defines the method <tt>__getstate__()</tt>, it is called and the
128  * return state is pickled as the contents for the instance, and if the class
129  * defines the method <tt>__setstate__()</tt>, it is called with the
130  * unpickled state. (Note that these methods can also be used to
131  * implement copying class instances.) If there is no
132  * <tt>__getstate__()</tt> method, the instance's <tt>__dict__</tt> is
133  * pickled. If there is no <tt>__setstate__()</tt> method, the pickled
134  * object must be a dictionary and its items are assigned to the new
135  * instance's dictionary. (If a class defines both <tt>__getstate__()</tt>
136  * and <tt>__setstate__()</tt>, the state object needn't be a dictionary
137  * -- these methods can do what they want.) This protocol is also used
138  * by the shallow and deep copying operations defined in the <tt>copy</tt>
139  * module.
140  * <P>
141  * Note that when class instances are pickled, their class's code and
142  * data are not pickled along with them. Only the instance data are
143  * pickled. This is done on purpose, so you can fix bugs in a class or
144  * add methods and still load objects that were created with an earlier
145  * version of the class. If you plan to have long-lived objects that
146  * will see many versions of a class, it may be worthwhile to put a version
147  * number in the objects so that suitable conversions can be made by the
148  * class's <tt>__setstate__()</tt> method.
149  *
150  * <P>
151  * When a class itself is pickled, only its name is pickled -- the class
152  * definition is not pickled, but re-imported by the unpickling process.
153  * Therefore, the restriction that the class must be defined at the top
154  * level in a module applies to pickled classes as well.
155  *
156  * <P>
157  *
158  * <P>
159  * The interface can be summarized as follows.
160  *
161  * <P>
162  * To pickle an object <tt>x</tt> onto a file <tt>f</tt>, open for writing:
163  *
164  * <P>
165  * <dl><dd><pre>
166  * p = pickle.Pickler(f)
167  * p.dump(x)
168  * </pre></dl>
169  *
170  * <P>
171  * A shorthand for this is:
172  *
173  * <P>
174  * <dl><dd><pre>
175  * pickle.dump(x, f)
176  * </pre></dl>
177  *
178  * <P>
179  * To unpickle an object <tt>x</tt> from a file <tt>f</tt>, open for reading:
180  *
181  * <P>
182  * <dl><dd><pre>
183  * u = pickle.Unpickler(f)
184  * x = u.load()
185  * </pre></dl>
186  *
187  * <P>
188  * A shorthand is:
189  *
190  * <P>
191  * <dl><dd><pre>
192  * x = pickle.load(f)
193  * </pre></dl>
194  *
195  * <P>
196  * The <tt>Pickler</tt> class only calls the method <tt>f.write()</tt> with a
197  * string argument. The <tt>Unpickler</tt> calls the methods
198  * <tt>f.read()</tt> (with an integer argument) and <tt>f.readline()</tt>
199  * (without argument), both returning a string. It is explicitly allowed to
200  * pass non-file objects here, as long as they have the right methods.
201  *
202  * <P>
203  * The constructor for the <tt>Pickler</tt> class has an optional second
204  * argument, <i>bin</i>. If this is present and nonzero, the binary
205  * pickle format is used; if it is zero or absent, the (less efficient,
206  * but backwards compatible) text pickle format is used. The
207  * <tt>Unpickler</tt> class does not have an argument to distinguish
208  * between binary and text pickle formats; it accepts either format.
209  *
210  * <P>
211  * The following types can be pickled:
212  *
213  * <UL><LI><tt>None</tt>
214  *
215  * <P>
216  *
217  * <LI>integers, long integers, floating point numbers
218  *
219  * <P>
220  *
221  * <LI>strings
222  *
223  * <P>
224  *
225  * <LI>tuples, lists and dictionaries containing only picklable objects
226  *
227  * <P>
228  *
229  * <LI>classes that are defined at the top level in a module
230  *
231  * <P>
232  *
233  * <LI>instances of such classes whose <tt>__dict__</tt> or
234  * <tt>__setstate__()</tt> is picklable
235  *
236  * <P>
237  *
238  * </UL>
239  *
240  * <P>
241  * Attempts to pickle unpicklable objects will raise the
242  * <tt>PicklingError</tt> exception; when this happens, an unspecified
243  * number of bytes may have been written to the file.
244  *
245  * <P>
246  * It is possible to make multiple calls to the <tt>dump()</tt> method of
247  * the same <tt>Pickler</tt> instance. These must then be matched to the
248  * same number of calls to the <tt>load()</tt> method of the
249  * corresponding <tt>Unpickler</tt> instance. If the same object is
250  * pickled by multiple <tt>dump()</tt> calls, the <tt>load()</tt> will all
251  * yield references to the same object. <i>Warning</i>: this is intended
252  * for pickling multiple objects without intervening modifications to the
253  * objects or their parts. If you modify an object and then pickle it
254  * again using the same <tt>Pickler</tt> instance, the object is not
255  * pickled again -- a reference to it is pickled and the
256  * <tt>Unpickler</tt> will return the old value, not the modified one.
257  * (There are two problems here: (a) detecting changes, and (b)
258  * marshalling a minimal set of changes. I have no answers. Garbage
259  * Collection may also become a problem here.)
260  *
261  * <P>
262  * Apart from the <tt>Pickler</tt> and <tt>Unpickler</tt> classes, the
263  * module defines the following functions, and an exception:
264  *
265  * <P>
266  * <dl><dt><b><tt>dump</tt></a></b> (<var>object, file</var><big>[</big><var>,
267  * bin</var><big>]</big>)
268  * <dd>
269  * Write a pickled representation of <i>obect</i> to the open file object
270  * <i>file</i>. This is equivalent to
271  * "<tt>Pickler(<i>file</i>, <i>bin</i>).dump(<i>object</i>)</tt>".
272  * If the optional <i>bin</i> argument is present and nonzero, the binary
273  * pickle format is used; if it is zero or absent, the (less efficient)
274  * text pickle format is used.
275  * </dl>
276  *
277  * <P>
278  * <dl><dt><b><tt>load</tt></a></b> (<var>file</var>)
279  * <dd>
280  * Read a pickled object from the open file object <i>file</i>. This is
281  * equivalent to "<tt>Unpickler(<i>file</i>).load()</tt>".
282  * </dl>
283  *
284  * <P>
285  * <dl><dt><b><tt>dumps</tt></a></b> (<var>object</var><big>[</big><var>,
286  * bin</var><big>]</big>)
287  * <dd>
288  * Return the pickled representation of the object as a string, instead
289  * of writing it to a file. If the optional <i>bin</i> argument is
290  * present and nonzero, the binary pickle format is used; if it is zero
291  * or absent, the (less efficient) text pickle format is used.
292  * </dl>
293  *
294  * <P>
295  * <dl><dt><b><tt>loads</tt></a></b> (<var>string</var>)
296  * <dd>
297  * Read a pickled object from a string instead of a file. Characters in
298  * the string past the pickled object's representation are ignored.
299  * </dl>
300  *
301  * <P>
302  * <dl><dt><b><a name="l2h-3763"><tt>PicklingError</tt></a></b>
303  * <dd>
304  * This exception is raised when an unpicklable object is passed to
305  * <tt>Pickler.dump()</tt>.
306  * </dl>
307  *
308  *
309  * <p>
310  * For the complete documentation on the pickle module, please see the
311  * "Python Library Reference"
312  * <p><hr><p>
313  *
314  * The module is based on both original pickle.py and the cPickle.c
315  * version, except that all mistakes and errors are my own.
316  * <p>
317  * @author Finn Bock, bckfnn@pipmail.dknet.dk
318  * @version cPickle.java,v 1.30 1999/05/15 17:40:12 fb Exp
319  */

320 public class cPickle implements ClassDictInit {
321     /**
322      * The doc string
323      */

324     public static String JavaDoc __doc__ =
325        "Java implementation and optimization of the Python pickle module\n" +
326        "\n" +
327        "$Id: cPickle.java,v 1.24 2005/02/22 04:19:33 bzimmer Exp $\n";
328
329
330     /**
331      * The program version.
332      */

333     public static String JavaDoc __version__ = "1.30";
334
335     /**
336      * File format version we write.
337      */

338     public static final String JavaDoc format_version = "1.3";
339
340     /**
341      * Old format versions we can read.
342      */

343     public static final String JavaDoc[] compatible_formats =
344                 new String JavaDoc[] { "1.0", "1.1", "1.2" };
345
346
347     public static String JavaDoc[] __depends__ = new String JavaDoc[] {
348         "copy_reg",
349     };
350
351     public static PyObject PickleError;
352     public static PyObject PicklingError;
353     public static PyObject UnpickleableError;
354     public static PyObject UnpicklingError;
355
356     public static final PyString BadPickleGet =
357                 new PyString("cPickle.BadPickleGet");
358
359
360     final static char MARK = '(';
361     final static char STOP = '.';
362     final static char POP = '0';
363     final static char POP_MARK = '1';
364     final static char DUP = '2';
365     final static char FLOAT = 'F';
366     final static char INT = 'I';
367     final static char BININT = 'J';
368     final static char BININT1 = 'K';
369     final static char LONG = 'L';
370     final static char BININT2 = 'M';
371     final static char NONE = 'N';
372     final static char PERSID = 'P';
373     final static char BINPERSID = 'Q';
374     final static char REDUCE = 'R';
375     final static char STRING = 'S';
376     final static char BINSTRING = 'T';
377     final static char SHORT_BINSTRING = 'U';
378     final static char UNICODE = 'V';
379     final static char BINUNICODE = 'X';
380     final static char APPEND = 'a';
381     final static char BUILD = 'b';
382     final static char GLOBAL = 'c';
383     final static char DICT = 'd';
384     final static char EMPTY_DICT = '}';
385     final static char APPENDS = 'e';
386     final static char GET = 'g';
387     final static char BINGET = 'h';
388     final static char INST = 'i';
389     final static char LONG_BINGET = 'j';
390     final static char LIST = 'l';
391     final static char EMPTY_LIST = ']';
392     final static char OBJ = 'o';
393     final static char PUT = 'p';
394     final static char BINPUT = 'q';
395     final static char LONG_BINPUT = 'r';
396     final static char SETITEM = 's';
397     final static char TUPLE = 't';
398     final static char EMPTY_TUPLE = ')';
399     final static char SETITEMS = 'u';
400     final static char BINFLOAT = 'G';
401
402     private static PyDictionary dispatch_table = null;
403     private static PyDictionary safe_constructors = null;
404
405
406     private static PyType BuiltinFunctionType =
407                             PyType.fromClass(PyReflectedFunction.class);
408     private static PyType BuiltinMethodType =
409                             PyType.fromClass(PyMethod.class);
410     private static PyType ClassType =
411                             PyType.fromClass(PyClass.class);
412     private static PyType DictionaryType =
413                             PyType.fromClass(PyDictionary.class);
414     private static PyType StringMapType =
415                             PyType.fromClass(PyStringMap.class);
416     private static PyType FloatType =
417                             PyType.fromClass(PyFloat.class);
418     private static PyType FunctionType =
419                             PyType.fromClass(PyFunction.class);
420     private static PyType InstanceType =
421                             PyType.fromClass(PyInstance.class);
422     private static PyType IntType =
423                             PyType.fromClass(PyInteger.class);
424     private static PyType ListType =
425                             PyType.fromClass(PyList.class);
426     private static PyType LongType =
427                             PyType.fromClass(PyLong.class);
428     private static PyType NoneType =
429                             PyType.fromClass(PyNone.class);
430     private static PyType StringType =
431                             PyType.fromClass(PyString.class);
432     private static PyType TupleType =
433                             PyType.fromClass(PyTuple.class);
434     private static PyType FileType =
435                             PyType.fromClass(PyFile.class);
436
437
438     private static PyObject dict;
439
440     /**
441      * Initialization when module is imported.
442      */

443     public static void classDictInit(PyObject dict) {
444         cPickle.dict = dict;
445
446         // XXX: Hack for JPython 1.0.1. By default __builtin__ is not in
447
// sys.modules.
448
imp.importName("__builtin__", true);
449
450         PyModule copyreg = (PyModule)importModule("copy_reg");
451
452         dispatch_table = (PyDictionary)copyreg.__getattr__("dispatch_table");
453         safe_constructors = (PyDictionary)
454                                     copyreg.__getattr__("safe_constructors");
455
456         PickleError = buildClass("PickleError", Py.Exception,
457                                        "_PickleError", "");
458         PicklingError = buildClass("PicklingError", PickleError,
459                                        "_empty__init__", "");
460         UnpickleableError = buildClass("UnpickleableError", PicklingError,
461                                        "_UnpickleableError", "");
462         UnpicklingError = buildClass("UnpicklingError", PickleError,
463                                        "_empty__init__", "");
464     }
465
466     // An empty __init__ method
467
public static PyObject _empty__init__(PyObject[] arg, String JavaDoc[] kws) {
468         PyObject dict = new PyStringMap();
469         dict.__setitem__("__module__", new PyString("cPickle"));
470         return dict;
471     }
472
473     public static PyObject _PickleError(PyObject[] arg, String JavaDoc[] kws) {
474         PyObject dict = _empty__init__(arg, kws);
475         dict.__setitem__("__init__", getJavaFunc("_PickleError__init__"));
476         dict.__setitem__("__str__", getJavaFunc("_PickleError__str__"));
477         return dict;
478     }
479
480     public static void _PickleError__init__(PyObject[] arg, String JavaDoc[] kws) {
481         ArgParser ap = new ArgParser("__init__", arg, kws, "self", "args");
482         PyObject self = ap.getPyObject(0);
483         PyObject args = ap.getList(1);
484
485         self.__setattr__("args", args);
486     }
487
488     public static PyString _PickleError__str__(PyObject[] arg, String JavaDoc[] kws) {
489         ArgParser ap = new ArgParser("__str__", arg, kws, "self");
490         PyObject self = ap.getPyObject(0);
491
492         PyObject args = self.__getattr__("args");
493         if (args.__len__() > 0 && args.__getitem__(0).__len__() > 0)
494             return args.__getitem__(0).__str__();
495         else
496             return new PyString("(what)");
497     }
498
499     public static PyObject _UnpickleableError(PyObject[] arg, String JavaDoc[] kws) {
500         PyObject dict = _empty__init__(arg, kws);
501         dict.__setitem__("__init__",
502                                 getJavaFunc("_UnpickleableError__init__"));
503         dict.__setitem__("__str__",
504                                 getJavaFunc("_UnpickleableError__str__"));
505         return dict;
506     }
507
508     public static void _UnpickleableError__init__(PyObject[] arg,
509                                                   String JavaDoc[] kws)
510     {
511         ArgParser ap = new ArgParser("__init__", arg, kws, "self", "args");
512         PyObject self = ap.getPyObject(0);
513         PyObject args = ap.getList(1);
514
515         self.__setattr__("args", args);
516     }
517
518     public static PyString _UnpickleableError__str__(PyObject[] arg,
519                                                      String JavaDoc[] kws)
520     {
521         ArgParser ap = new ArgParser("__str__", arg, kws, "self");
522         PyObject self = ap.getPyObject(0);
523
524         PyObject args = self.__getattr__("args");
525         PyObject a = args.__len__() > 0 ? args.__getitem__(0) :
526                                 new PyString("(what)");
527         return new PyString("Cannot pickle %s objects").__mod__(a).__str__();
528     }
529
530
531     public cPickle() {
532     }
533
534
535     /**
536      * Returns a pickler instance.
537      * @param file a file-like object, can be a cStringIO.StringIO,
538      * a PyFile or any python object which implements a
539      * <i>write</i> method. The data will be written as text.
540      * @returns a new Pickler instance.
541      */

542     public static Pickler Pickler(PyObject file) {
543         return new Pickler(file, false);
544     }
545
546
547     /**
548      * Returns a pickler instance.
549      * @param file a file-like object, can be a cStringIO.StringIO,
550      * a PyFile or any python object which implements a
551      * <i>write</i> method.
552      * @param bin when true, the output will be written as binary data.
553      * @returns a new Pickler instance.
554      */

555     public static Pickler Pickler(PyObject file, boolean bin) {
556         return new Pickler(file, bin);
557     }
558
559
560     /**
561      * Returns a unpickler instance.
562      * @param file a file-like object, can be a cStringIO.StringIO,
563      * a PyFile or any python object which implements a
564      * <i>read</i> and <i>readline</i> method.
565      * @returns a new Unpickler instance.
566      */

567     public static Unpickler Unpickler(PyObject file) {
568         return new Unpickler(file);
569     }
570
571
572     /**
573      * Shorthand function which pickles the object on the file.
574      * @param object a data object which should be pickled.
575      * @param file a file-like object, can be a cStringIO.StringIO,
576      * a PyFile or any python object which implements a
577      * <i>write</i> method. The data will be written as
578      * text.
579      * @returns a new Unpickler instance.
580      */

581     public static void dump(PyObject object, PyObject file) {
582         dump(object, file, false);
583     }
584
585     /**
586      * Shorthand function which pickles the object on the file.
587      * @param object a data object which should be pickled.
588      * @param file a file-like object, can be a cStringIO.StringIO,
589      * a PyFile or any python object which implements a
590      * <i>write</i> method.
591      * @param bin when true, the output will be written as binary data.
592      * @returns a new Unpickler instance.
593      */

594     public static void dump(PyObject object, PyObject file, boolean bin) {
595         new Pickler(file, bin).dump(object);
596     }
597
598
599     /**
600      * Shorthand function which pickles and returns the string representation.
601      * @param object a data object which should be pickled.
602      * @returns a string representing the pickled object.
603      */

604     public static String JavaDoc dumps(PyObject object) {
605         return dumps(object, false);
606     }
607
608
609     /**
610      * Shorthand function which pickles and returns the string representation.
611      * @param object a data object which should be pickled.
612      * @param bin when true, the output will be written as binary data.
613      * @returns a string representing the pickled object.
614      */

615     public static String JavaDoc dumps(PyObject object, boolean bin) {
616         cStringIO.StringIO file = cStringIO.StringIO();
617         dump(object, file, bin);
618         return file.getvalue();
619     }
620
621
622     /**
623      * Shorthand function which unpickles a object from the file and returns
624      * the new object.
625      * @param file a file-like object, can be a cStringIO.StringIO,
626      * a PyFile or any python object which implements a
627      * <i>read</i> and <i>readline</i> method.
628      * @returns a new object.
629      */

630     public static Object JavaDoc load(PyObject file) {
631         return new Unpickler(file).load();
632     }
633
634
635     /**
636      * Shorthand function which unpickles a object from the string and
637      * returns the new object.
638      * @param str a strings which must contain a pickled object
639      * representation.
640      * @returns a new object.
641      */

642     public static Object JavaDoc loads(PyObject str) {
643         cStringIO.StringIO file = cStringIO.StringIO(str.toString());
644         return new Unpickler(file).load();
645     }
646
647
648
649     // Factory for creating IOFile representation.
650
private static IOFile createIOFile(PyObject file) {
651         Object JavaDoc f = file.__tojava__(cStringIO.StringIO.class);
652         if (f != Py.NoConversion)
653             return new cStringIOFile((cStringIO.StringIO)file);
654         else if (__builtin__.isinstance(file, FileType))
655             return new FileIOFile(file);
656         else
657             return new ObjectIOFile(file);
658     }
659
660
661     // IOFiles encapsulates and optimise access to the different file
662
// representation.
663
interface IOFile {
664         public abstract void write(String JavaDoc str);
665         // Usefull optimization since most data written are chars.
666
public abstract void write(char str);
667         public abstract void flush();
668         public abstract String JavaDoc read(int len);
669         // Usefull optimization since all readlines removes the
670
// trainling newline.
671
public abstract String JavaDoc readlineNoNl();
672
673     }
674
675
676     // Use a cStringIO as a file.
677
static class cStringIOFile implements IOFile {
678         cStringIO.StringIO file;
679
680         cStringIOFile(PyObject file) {
681             this.file = (cStringIO.StringIO)file.__tojava__(Object JavaDoc.class);
682         }
683
684         public void write(String JavaDoc str) {
685             file.write(str);
686         }
687
688         public void write(char ch) {
689             file.writeChar(ch);
690         }
691
692         public void flush() {}
693
694         public String JavaDoc read(int len) {
695             return file.read(len);
696         }
697
698         public String JavaDoc readlineNoNl() {
699             return file.readlineNoNl();
700         }
701     }
702
703
704     // Use a PyFile as a file.
705
static class FileIOFile implements IOFile {
706         PyFile file;
707
708         FileIOFile(PyObject file) {
709             this.file = (PyFile)file.__tojava__(PyFile.class);
710             if (this.file.closed)
711                 throw Py.ValueError("I/O operation on closed file");
712         }
713
714         public void write(String JavaDoc str) {
715             file.write(str);
716         }
717
718         public void write(char ch) {
719             file.write(cStringIO.getString(ch));
720         }
721
722         public void flush() {}
723
724         public String JavaDoc read(int len) {
725             return file.read(len).toString();
726         }
727
728         public String JavaDoc readlineNoNl() {
729             String JavaDoc line = file.readline().toString();
730             return line.substring(0, line.length()-1);
731         }
732     }
733
734
735     // Use any python object as a file.
736
static class ObjectIOFile implements IOFile {
737         char[] charr = new char[1];
738         StringBuffer JavaDoc buff = new StringBuffer JavaDoc();
739         PyObject write;
740         PyObject read;
741         PyObject readline;
742         final int BUF_SIZE = 256;
743
744         ObjectIOFile(PyObject file) {
745 // this.file = file;
746
write = file.__findattr__("write");
747             read = file.__findattr__("read");
748             readline = file.__findattr__("readline");
749         }
750
751         public void write(String JavaDoc str) {
752             buff.append(str);
753             if (buff.length() > BUF_SIZE)
754                 flush();
755         }
756
757         public void write(char ch) {
758             buff.append(ch);
759             if (buff.length() > BUF_SIZE)
760                 flush();
761         }
762
763         public void flush() {
764             write.__call__(new PyString(buff.toString()));
765             buff.setLength(0);
766         }
767
768         public String JavaDoc read(int len) {
769             return read.__call__(new PyInteger(len)).toString();
770         }
771
772         public String JavaDoc readlineNoNl() {
773             String JavaDoc line = readline.__call__().toString();
774             return line.substring(0, line.length()-1);
775         }
776     }
777
778
779     /**
780      * The Pickler object
781      * @see cPickle#Pickler(PyObject)
782      * @see cPickle#Pickler(PyObject,boolean)
783      */

784     static public class Pickler {
785         private IOFile file;
786         private boolean bin;
787
788         /**
789          * Hmm, not documented, perhaps it shouldn't be public? XXX: fixme.
790          */

791         private PickleMemo memo = new PickleMemo();
792
793         /**
794          * To write references to persistent objects, the persistent module
795          * must assign a method to persistent_id which returns either None
796          * or the persistent ID of the object.
797          * For the benefit of persistency modules written using pickle,
798          * it supports the notion of a reference to an object outside
799          * the pickled data stream.
800          * Such objects are referenced by a name, which is an arbitrary
801          * string of printable ASCII characters.
802          */

803         public PyObject persistent_id = null;
804
805         /**
806          * Hmm, not documented, perhaps it shouldn't be public? XXX: fixme.
807          */

808         public PyObject inst_persistent_id = null;
809
810
811         public Pickler(PyObject file, boolean bin) {
812             this.file = createIOFile(file);
813             this.bin = bin;
814         }
815
816
817         /**
818          * Write a pickled representation of the object.
819          * @param object The object which will be pickled.
820          */

821         public void dump(PyObject object) {
822             save(object);
823             file.write(STOP);
824             file.flush();
825         }
826
827         private static final int get_id(PyObject o) {
828             // we don't pickle Java instances so we don't have to consider that case
829
return System.identityHashCode(o);
830         }
831
832
833         // Save name as in pickle.py but semantics are slightly changed.
834
private void put(int i) {
835             if (bin) {
836                 if (i < 256) {
837                     file.write(BINPUT);
838                     file.write((char)i);
839                     return;
840                 }
841                 file.write(LONG_BINPUT);
842                 file.write((char)( i & 0xFF));
843                 file.write((char)((i >>> 8) & 0xFF));
844                 file.write((char)((i >>> 16) & 0xFF));
845                 file.write((char)((i >>> 24) & 0xFF));
846                 return;
847             }
848             file.write(PUT);
849             file.write(String.valueOf(i));
850             file.write("\n");
851         }
852
853
854         // Same name as in pickle.py but semantics are slightly changed.
855
private void get(int i) {
856             if (bin) {
857                 if (i < 256) {
858                     file.write(BINGET);
859                     file.write((char)i);
860                     return;
861                 }
862                 file.write(LONG_BINGET);
863                 file.write((char)( i & 0xFF));
864                 file.write((char)((i >>> 8) & 0xFF));
865                 file.write((char)((i >>> 16) & 0xFF));
866                 file.write((char)((i >>> 24) & 0xFF));
867                 return;
868             }
869             file.write(GET);
870             file.write(String.valueOf(i));
871             file.write("\n");
872         }
873
874
875         private void save(PyObject object) {
876             save(object, false);
877         }
878
879
880         private void save(PyObject object, boolean pers_save) {
881             if (!pers_save) {
882                 if (persistent_id != null) {
883                     PyObject pid = persistent_id.__call__(object);
884                     if (pid != Py.None) {
885                         save_pers(pid);
886                         return;
887                     }
888                 }
889             }
890
891             int d = get_id(object);
892
893             PyType t = object.getType();
894
895             if (t == TupleType && object.__len__() == 0) {
896                 if (bin)
897                     save_empty_tuple(object);
898                 else
899                     save_tuple(object);
900                 return;
901             }
902
903             int m = getMemoPosition(d, object);
904             if (m >= 0) {
905                 get(m);
906                 return;
907             }
908
909             if (save_type(object, t))
910                 return;
911
912             if (inst_persistent_id != null) {
913                 PyObject pid = inst_persistent_id.__call__(object);
914                 if (pid != Py.None) {
915                     save_pers(pid);
916                     return;
917                 }
918             }
919
920             PyObject tup = null;
921             PyObject reduce = dispatch_table.__finditem__(t);
922             if (reduce == null) {
923                 reduce = object.__findattr__("__reduce__");
924                 if (reduce == null)
925                     throw new PyException(UnpickleableError, object);
926                 tup = reduce.__call__();
927             } else {
928                 tup = reduce.__call__(object);
929             }
930
931             if (tup instanceof PyString) {
932                 save_global(object, tup);
933                 return;
934             }
935
936             if (!(tup instanceof PyTuple)) {
937                 throw new PyException(PicklingError,
938                             "Value returned by " + reduce.__repr__() +
939                             " must be a tuple");
940             }
941
942             int l = tup.__len__();
943             if (l != 2 && l != 3) {
944                 throw new PyException(PicklingError,
945                             "tuple returned by " + reduce.__repr__() +
946                             " must contain only two or three elements");
947             }
948
949             PyObject callable = tup.__finditem__(0);
950             PyObject arg_tup = tup.__finditem__(1);
951             PyObject state = (l > 2) ? tup.__finditem__(2) : Py.None;
952
953             if (!(arg_tup instanceof PyTuple) && arg_tup != Py.None) {
954                 throw new PyException(PicklingError,
955                             "Second element of tupe returned by " +
956                             reduce.__repr__() + " must be a tuple");
957             }
958
959             save_reduce(callable, arg_tup, state);
960
961             put(putMemo(d, object));
962         }
963
964
965         final private void save_pers(PyObject pid) {
966             if (!bin) {
967                 file.write(PERSID);
968                 file.write(pid.toString());
969                 file.write("\n");
970             } else {
971                 save(pid, true);
972                 file.write(BINPERSID);
973             }
974         }
975
976         final private void save_reduce(PyObject callable, PyObject arg_tup,
977                                        PyObject state)
978         {
979             save(callable);
980             save(arg_tup);
981             file.write(REDUCE);
982             if (state != Py.None) {
983                 save(state);
984                 file.write(BUILD);
985             }
986         }
987
988
989
990         final private boolean save_type(PyObject object, PyType type) {
991             //System.out.println("save_type " + object + " " + cls);
992
if (type == NoneType)
993                 save_none(object);
994             else if (type == StringType)
995                 save_string(object);
996             else if (type == IntType)
997                 save_int(object);
998             else if (type == LongType)
999                 save_long(object);
1000            else if (type == FloatType)
1001                save_float(object);
1002            else if (type == TupleType)
1003                save_tuple(object);
1004            else if (type == ListType)
1005                save_list(object);
1006            else if (type == DictionaryType || type == StringMapType)
1007                save_dict(object);
1008            else if (type == InstanceType)
1009                save_inst((PyInstance)object);
1010            else if (type == ClassType)
1011                save_global(object);
1012            else if (type == FunctionType)
1013                save_global(object);
1014            else if (type == BuiltinFunctionType)
1015                save_global(object);
1016            else
1017                return false;
1018            return true;
1019        }
1020
1021
1022
1023        final private void save_none(PyObject object) {
1024            file.write(NONE);
1025        }
1026
1027        final private void save_int(PyObject object) {
1028            if (bin) {
1029                int l = ((PyInteger)object).getValue();
1030                char i1 = (char)( l & 0xFF);
1031                char i2 = (char)((l >>> 8 ) & 0xFF);
1032                char i3 = (char)((l >>> 16) & 0xFF);
1033                char i4 = (char)((l >>> 24) & 0xFF);
1034
1035                if (i3 == '\0' && i4 == '\0') {
1036                    if (i2 == '\0') {
1037                        file.write(BININT1);
1038                        file.write(i1);
1039                        return;
1040                    }
1041                    file.write(BININT2);
1042                    file.write(i1);
1043                    file.write(i2);
1044                    return;
1045                }
1046                file.write(BININT);
1047                file.write(i1);
1048                file.write(i2);
1049                file.write(i3);
1050                file.write(i4);
1051            } else {
1052                file.write(INT);
1053                file.write(object.toString());
1054                file.write("\n");
1055            }
1056        }
1057
1058
1059        final private void save_long(PyObject object) {
1060            file.write(LONG);
1061            file.write(object.toString());
1062            file.write("\n");
1063        }
1064
1065
1066        final private void save_float(PyObject object) {
1067            if (bin) {
1068                file.write(BINFLOAT);
1069                double value= ((PyFloat) object).getValue();
1070                // It seems that struct.pack('>d', ..) and doubleToLongBits
1071
// are the same. Good for me :-)
1072
long bits = Double.doubleToLongBits(value);
1073                file.write((char)((bits >>> 56) & 0xFF));
1074                file.write((char)((bits >>> 48) & 0xFF));
1075                file.write((char)((bits >>> 40) & 0xFF));
1076                file.write((char)((bits >>> 32) & 0xFF));
1077                file.write((char)((bits >>> 24) & 0xFF));
1078                file.write((char)((bits >>> 16) & 0xFF));
1079                file.write((char)((bits >>> 8) & 0xFF));
1080                file.write((char)((bits >>> 0) & 0xFF));
1081            } else {
1082                file.write(FLOAT);
1083                file.write(object.toString());
1084                file.write("\n");
1085            }
1086        }
1087
1088
1089        final private void save_string(PyObject object) {
1090            boolean unicode = ((PyString) object).isunicode();
1091            String JavaDoc str = object.toString();
1092
1093            if (bin) {
1094                if (unicode)
1095                    str = codecs.PyUnicode_EncodeUTF8(str, "struct");
1096                int l = str.length();
1097                if (l < 256 && !unicode) {
1098                    file.write(SHORT_BINSTRING);
1099                    file.write((char)l);
1100                } else {
1101                    if (unicode)
1102                        file.write(BINUNICODE);
1103                    else
1104                        file.write(BINSTRING);
1105                    file.write((char)( l & 0xFF));
1106                    file.write((char)((l >>> 8 ) & 0xFF));
1107                    file.write((char)((l >>> 16) & 0xFF));
1108                    file.write((char)((l >>> 24) & 0xFF));
1109                }
1110                file.write(str);
1111            } else {
1112                if (unicode) {
1113                    file.write(UNICODE);
1114                    file.write(codecs.PyUnicode_EncodeRawUnicodeEscape(str,
1115                                                            "strict", true));
1116                } else {
1117                    file.write(STRING);
1118                    file.write(object.__repr__().toString());
1119                }
1120                file.write("\n");
1121            }
1122            put(putMemo(get_id(object), object));
1123        }
1124
1125
1126        final private void save_tuple(PyObject object) {
1127            int d = get_id(object);
1128
1129            file.write(MARK);
1130
1131            int len = object.__len__();
1132
1133            for (int i = 0; i < len; i++)
1134                save(object.__finditem__(i));
1135
1136            if (len > 0) {
1137                int m = getMemoPosition(d, object);
1138                if (m >= 0) {
1139                    if (bin) {
1140                        file.write(POP_MARK);
1141                        get(m);
1142                        return;
1143                    }
1144                    for (int i = 0; i < len+1; i++)
1145                        file.write(POP);
1146                    get(m);
1147                    return;
1148                }
1149            }
1150            file.write(TUPLE);
1151            put(putMemo(d, object));
1152        }
1153
1154
1155        final private void save_empty_tuple(PyObject object) {
1156            file.write(EMPTY_TUPLE);
1157        }
1158
1159        final private void save_list(PyObject object) {
1160            if (bin)
1161                file.write(EMPTY_LIST);
1162            else {
1163                file.write(MARK);
1164                file.write(LIST);
1165            }
1166
1167            put(putMemo(get_id(object), object));
1168
1169            int len = object.__len__();
1170            boolean using_appends = bin && len > 1;
1171
1172            if (using_appends)
1173                file.write(MARK);
1174
1175            for (int i = 0; i < len; i++) {
1176                save(object.__finditem__(i));
1177                if (!using_appends)
1178                    file.write(APPEND);
1179            }
1180            if (using_appends)
1181                file.write(APPENDS);
1182        }
1183
1184
1185        final private void save_dict(PyObject object) {
1186            if (bin)
1187                file.write(EMPTY_DICT);
1188            else {
1189                file.write(MARK);
1190                file.write(DICT);
1191            }
1192
1193            put(putMemo(get_id(object), object));
1194
1195            PyObject list = object.invoke("keys");
1196            int len = list.__len__();
1197
1198            boolean using_setitems = (bin && len > 1);
1199
1200            if (using_setitems)
1201                file.write(MARK);
1202
1203            for (int i = 0; i < len; i++) {
1204                PyObject key = list.__finditem__(i);
1205                PyObject value = object.__finditem__(key);
1206                save(key);
1207                save(value);
1208
1209                if (!using_setitems)
1210                     file.write(SETITEM);
1211            }
1212            if (using_setitems)
1213                file.write(SETITEMS);
1214        }
1215
1216
1217        final private void save_inst(PyInstance object) {
1218            if (object instanceof PyJavaInstance)
1219                throw new PyException(PicklingError,
1220                            "Unable to pickle java objects.");
1221
1222            PyClass cls = object.instclass;
1223
1224            PySequence args = null;
1225            PyObject getinitargs = object.__findattr__("__getinitargs__");
1226            if (getinitargs != null) {
1227                args = (PySequence)getinitargs.__call__();
1228                // XXX Assert it's a sequence
1229
keep_alive(args);
1230            }
1231
1232            file.write(MARK);
1233            if (bin)
1234                save(cls);
1235
1236            if (args != null) {
1237                int len = args.__len__();
1238                for (int i = 0; i < len; i++)
1239                    save(args.__finditem__(i));
1240            }
1241
1242            int mid = putMemo(get_id(object), object);
1243            if (bin) {
1244                file.write(OBJ);
1245                put(mid);
1246            } else {
1247                file.write(INST);
1248                file.write(cls.__findattr__("__module__").toString());
1249                file.write("\n");
1250                file.write(cls.__name__);
1251                file.write("\n");
1252                put(mid);
1253            }
1254
1255            PyObject stuff = null;
1256            PyObject getstate = object.__findattr__("__getstate__");
1257            if (getstate == null) {
1258                stuff = object.__dict__;
1259            } else {
1260                stuff = getstate.__call__();
1261                keep_alive(stuff);
1262            }
1263            save(stuff);
1264            file.write(BUILD);
1265        }
1266
1267
1268        final private void save_global(PyObject object) {
1269            save_global(object, null);
1270        }
1271
1272
1273        final private void save_global(PyObject object, PyObject name) {
1274            if (name == null)
1275                name = object.__findattr__("__name__");
1276
1277            PyObject module = object.__findattr__("__module__");
1278            if (module == null || module == Py.None)
1279                module = whichmodule(object, name);
1280
1281            file.write(GLOBAL);
1282            file.write(module.toString());
1283            file.write("\n");
1284            file.write(name.toString());
1285            file.write("\n");
1286            put(putMemo(get_id(object), object));
1287        }
1288
1289
1290        final private int getMemoPosition(int id, Object JavaDoc o) {
1291            return memo.findPosition(id, o);
1292        }
1293
1294        final private int putMemo(int id, PyObject object) {
1295            int memo_len = memo.size() + 1;
1296            memo.put(id, memo_len, object);
1297            return memo_len;
1298        }
1299
1300
1301        /**
1302         * Keeps a reference to the object x in the memo.
1303         *
1304         * Because we remember objects by their id, we have
1305         * to assure that possibly temporary objects are kept
1306         * alive by referencing them.
1307         * We store a reference at the id of the memo, which should
1308         * normally not be used unless someone tries to deepcopy
1309         * the memo itself...
1310         */

1311        final private void keep_alive(PyObject obj) {
1312            int id = System.identityHashCode(memo);
1313            PyList list = (PyList) memo.findValue(id, memo);
1314            if (list == null) {
1315                list = new PyList();
1316                memo.put(id, -1, list);
1317            }
1318            list.append(obj);
1319        }
1320    }
1321
1322
1323
1324
1325    private static Hashtable classmap = new Hashtable();
1326
1327    final private static PyObject whichmodule(PyObject cls,
1328                                              PyObject clsname)
1329    {
1330        PyObject name = (PyObject)classmap.get(cls);
1331        if (name != null)
1332            return name;
1333
1334        name = new PyString("__main__");
1335
1336        // For use with JPython1.0.x
1337
//PyObject modules = sys.modules;
1338

1339        // For use with JPython1.1.x
1340
//PyObject modules = Py.getSystemState().modules;
1341

1342        PyObject sys = imp.importName("sys", true);
1343        PyObject modules = sys.__findattr__("modules");
1344        PyObject keylist = modules.invoke("keys");
1345
1346        int len = keylist.__len__();
1347        for (int i = 0; i < len; i++) {
1348            PyObject key = keylist.__finditem__(i);
1349            PyObject value = modules.__finditem__(key);
1350
1351            if (!key.equals("__main__") &&
1352                    value.__findattr__(clsname.toString().intern()) == cls) {
1353                name = key;
1354                break;
1355            }
1356        }
1357
1358        classmap.put(cls, name);
1359        //System.out.println(name);
1360
return name;
1361    }
1362
1363
1364    /*
1365     * A very specialized and simplified version of PyStringMap. It can
1366     * only use integers as keys and stores both an integer and an object
1367     * as value. It is very private!
1368     */

1369    static private class PickleMemo {
1370        //Table of primes to cycle through
1371
private final int[] primes = {
1372            13, 61, 251, 1021, 4093,
1373            5987, 9551, 15683, 19609, 31397,
1374            65521, 131071, 262139, 524287, 1048573, 2097143,
1375            4194301, 8388593, 16777213, 33554393, 67108859,
1376            134217689, 268435399, 536870909, 1073741789,};
1377
1378        private transient int[] keys;
1379        private transient int[] position;
1380        private transient Object JavaDoc[] values;
1381
1382        private int size;
1383        private transient int filled;
1384        private transient int prime;
1385
1386        public PickleMemo(int capacity) {
1387            prime = 0;
1388            keys = null;
1389            values = null;
1390            resize(capacity);
1391        }
1392
1393        public PickleMemo() {
1394            this(4);
1395        }
1396
1397        public synchronized int size() {
1398            return size;
1399        }
1400
1401        private int findIndex(int key, Object JavaDoc value) {
1402            int[] table = keys;
1403            int maxindex = table.length;
1404            int index = (key & 0x7fffffff) % maxindex;
1405
1406            // Fairly aribtrary choice for stepsize...
1407
int stepsize = maxindex / 5;
1408
1409            // Cycle through possible positions for the key;
1410
//int collisions = 0;
1411
while (true) {
1412                int tkey = table[index];
1413                if (tkey == key && value == values[index]) {
1414                    return index;
1415                }
1416                if (values[index] == null) return -1;
1417                index = (index+stepsize) % maxindex;
1418            }
1419        }
1420
1421        public int findPosition(int key, Object JavaDoc value) {
1422            int idx = findIndex(key, value);
1423            if (idx < 0) return -1;
1424            return position[idx];
1425        }
1426
1427
1428        public Object JavaDoc findValue(int key, Object JavaDoc value) {
1429            int idx = findIndex(key, value);
1430            if (idx < 0) return null;
1431            return values[idx];
1432        }
1433
1434
1435        private final void insertkey(int key, int pos, Object JavaDoc value) {
1436            int[] table = keys;
1437            int maxindex = table.length;
1438            int index = (key & 0x7fffffff) % maxindex;
1439
1440            // Fairly aribtrary choice for stepsize...
1441
int stepsize = maxindex / 5;
1442
1443            // Cycle through possible positions for the key;
1444
while (true) {
1445                int tkey = table[index];
1446                if (values[index] == null) {
1447                    table[index] = key;
1448                    position[index] = pos;
1449                    values[index] = value;
1450                    filled++;
1451                    size++;
1452                    break;
1453                } else if (tkey == key && values[index] == value) {
1454                    position[index] = pos;
1455                    break;
1456                }
1457                index = (index+stepsize) % maxindex;
1458            }
1459        }
1460
1461
1462        private synchronized final void resize(int capacity) {
1463            int p = prime;
1464            for(; p<primes.length; p++) {
1465                if (primes[p] >= capacity) break;
1466            }
1467            if (primes[p] < capacity) {
1468                throw Py.ValueError("can't make hashtable of size: " +
1469                                    capacity);
1470            }
1471            capacity = primes[p];
1472            prime = p;
1473
1474            int[] oldKeys = keys;
1475            int[] oldPositions = position;
1476            Object JavaDoc[] oldValues = values;
1477
1478            keys = new int[capacity];
1479            position = new int[capacity];
1480            values = new Object JavaDoc[capacity];
1481            size = 0;
1482            filled = 0;
1483
1484            if (oldValues != null) {
1485                int n = oldValues.length;
1486
1487                for(int i=0; i<n; i++) {
1488                    Object JavaDoc value = oldValues[i];
1489                    if (value == null) continue;
1490                    insertkey(oldKeys[i], oldPositions[i], value);
1491                }
1492            }
1493        }
1494
1495        public void put(int key, int pos, Object JavaDoc value) {
1496            if (2*filled > keys.length) resize(keys.length+1);
1497            insertkey(key, pos, value);
1498        }
1499    }
1500
1501
1502
1503
1504
1505
1506    /**
1507     * The Unpickler object. Unpickler instances are create by the factory
1508     * methods Unpickler.
1509     * @see cPickle#Unpickler(PyObject)
1510     */

1511    static public class Unpickler {
1512
1513        private IOFile file;
1514
1515        public Hashtable memo = new Hashtable();
1516
1517        /**
1518         * For the benefit of persistency modules written using pickle,
1519         * it supports the notion of a reference to an object outside
1520         * the pickled data stream.
1521         * Such objects are referenced by a name, which is an arbitrary
1522         * string of printable ASCII characters.
1523         * The resolution of such names is not defined by the pickle module
1524         * -- the persistent object module will have to add a method
1525         * persistent_load().
1526         */

1527        public PyObject persistent_load = null;
1528
1529        private PyObject mark = new PyString("spam");
1530
1531        private int stackTop;
1532        private PyObject[] stack;
1533
1534
1535        Unpickler(PyObject file) {
1536            this.file = createIOFile(file);
1537        }
1538
1539
1540        /**
1541         * Unpickle and return an instance of the object represented by
1542         * the file.
1543         */

1544        public PyObject load() {
1545            stackTop = 0;
1546            stack = new PyObject[10];
1547
1548            while (true) {
1549                String JavaDoc s = file.read(1);
1550// System.out.println("load:" + s);
1551
// for (int i = 0; i < stackTop; i++)
1552
// System.out.println(" " + stack[i]);
1553
if (s.length() < 1)
1554                    load_eof();
1555                char key = s.charAt(0);
1556                switch (key) {
1557                case PERSID: load_persid(); break;
1558                case BINPERSID: load_binpersid(); break;
1559                case NONE: load_none(); break;
1560                case INT: load_int(); break;
1561                case BININT: load_binint(); break;
1562                case BININT1: load_binint1(); break;
1563                case BININT2: load_binint2(); break;
1564                case LONG: load_long(); break;
1565                case FLOAT: load_float(); break;
1566                case BINFLOAT: load_binfloat(); break;
1567                case STRING: load_string(); break;
1568                case BINSTRING: load_binstring(); break;
1569                case SHORT_BINSTRING: load_short_binstring(); break;
1570                case UNICODE: load_unicode(); break;
1571                case BINUNICODE: load_binunicode(); break;
1572                case TUPLE: load_tuple(); break;
1573                case EMPTY_TUPLE: load_empty_tuple(); break;
1574                case EMPTY_LIST: load_empty_list(); break;
1575                case EMPTY_DICT: load_empty_dictionary(); break;
1576                case LIST: load_list(); break;
1577                case DICT: load_dict(); break;
1578                case INST: load_inst(); break;
1579                case OBJ: load_obj(); break;
1580                case GLOBAL: load_global(); break;
1581                case REDUCE: load_reduce(); break;
1582                case POP: load_pop(); break;
1583                case POP_MARK: load_pop_mark(); break;
1584                case DUP: load_dup(); break;
1585                case GET: load_get(); break;
1586                case BINGET: load_binget(); break;
1587                case LONG_BINGET: load_long_binget(); break;
1588                case PUT: load_put(); break;
1589                case BINPUT: load_binput(); break;
1590                case LONG_BINPUT: load_long_binput(); break;
1591                case APPEND: load_append(); break;
1592                case APPENDS: load_appends(); break;
1593                case SETITEM: load_setitem(); break;
1594                case SETITEMS: load_setitems(); break;
1595                case BUILD: load_build(); break;
1596                case MARK: load_mark(); break;
1597                case STOP:
1598                    return load_stop();
1599                }
1600            }
1601        }
1602
1603
1604        final private int marker() {
1605            for (int k = stackTop-1; k >= 0; k--)
1606                if (stack[k] == mark)
1607                    return stackTop-k-1;
1608            throw new PyException(UnpicklingError,
1609                        "Inputstream corrupt, marker not found");
1610        }
1611
1612
1613        final private void load_eof() {
1614            throw new PyException(Py.EOFError);
1615        }
1616
1617
1618        final private void load_persid() {
1619            String JavaDoc pid = file.readlineNoNl();
1620            push(persistent_load.__call__(new PyString(pid)));
1621        }
1622
1623
1624        final private void load_binpersid() {
1625            PyObject pid = pop();
1626            push(persistent_load.__call__(pid));
1627        }
1628
1629
1630        final private void load_none() {
1631            push(Py.None);
1632        }
1633
1634
1635        final private void load_int() {
1636            String JavaDoc line = file.readlineNoNl();
1637            // XXX: use strop instead?
1638
push(new PyInteger(Integer.parseInt(line)));
1639        }
1640
1641
1642        final private void load_binint() {
1643            String JavaDoc s = file.read(4);
1644            int x = s.charAt(0) |
1645                   (s.charAt(1)<<8) |
1646                   (s.charAt(2)<<16) |
1647                   (s.charAt(3)<<24);
1648            push(new PyInteger(x));
1649        }
1650
1651
1652        final private void load_binint1() {
1653            int val = (int)file.read(1).charAt(0);
1654            push(new PyInteger(val));
1655        }
1656
1657        final private void load_binint2() {
1658            String JavaDoc s = file.read(2);
1659            int val = ((int)s.charAt(1)) << 8 | ((int)s.charAt(0));
1660            push(new PyInteger(val));
1661        }
1662
1663
1664        final private void load_long() {
1665            String JavaDoc line = file.readlineNoNl();
1666            push(new PyLong(line.substring(0, line.length()-1)));
1667        }
1668
1669        final private void load_float() {
1670            String JavaDoc line = file.readlineNoNl();
1671            push(new PyFloat(Double.valueOf(line).doubleValue()));
1672        }
1673
1674        final private void load_binfloat() {
1675            String JavaDoc s = file.read(8);
1676            long bits = (long)s.charAt(7) |
1677                        ((long)s.charAt(6) << 8) |
1678                        ((long)s.charAt(5) << 16) |
1679                        ((long)s.charAt(4) << 24) |
1680                        ((long)s.charAt(3) << 32) |
1681                        ((long)s.charAt(2) << 40) |
1682                        ((long)s.charAt(1) << 48) |
1683                        ((long)s.charAt(0) << 56);
1684            push(new PyFloat(Double.longBitsToDouble(bits)));
1685        }
1686
1687        final private void load_string() {
1688            String JavaDoc line = file.readlineNoNl();
1689
1690            String JavaDoc value;
1691            char quote = line.charAt(0);
1692            if (quote != '"' && quote != '\'')
1693                throw Py.ValueError("insecure string pickle");
1694
1695            int nslash = 0;
1696            int i;
1697            char ch = '\0';
1698            int n = line.length();
1699            for (i = 1; i < n; i++) {
1700                ch = line.charAt(i);
1701                if (ch == quote && nslash % 2 == 0)
1702                    break;
1703                if (ch == '\\')
1704                    nslash++;
1705                else
1706                    nslash = 0;
1707            }
1708            if (ch != quote)
1709                throw Py.ValueError("insecure string pickle");
1710
1711            for (i++ ; i < line.length(); i++) {
1712                if (line.charAt(i) > ' ')
1713                    throw Py.ValueError("insecure string pickle " + i);
1714            }
1715            value = PyString.decode_UnicodeEscape(line, 1, n-1,
1716                                                  "strict", false);
1717
1718            push(new PyString(value));
1719        }
1720
1721
1722        final private void load_binstring() {
1723            String JavaDoc d = file.read(4);
1724            int len = d.charAt(0) |
1725                     (d.charAt(1)<<8) |
1726                     (d.charAt(2)<<16) |
1727                     (d.charAt(3)<<24);
1728            push(new PyString(file.read(len)));
1729        }
1730
1731
1732        final private void load_short_binstring() {
1733            int len = (int)file.read(1).charAt(0);
1734            push(new PyString(file.read(len)));
1735        }
1736
1737
1738        final private void load_unicode() {
1739            String JavaDoc line = file.readlineNoNl();
1740            int n = line.length();
1741            String JavaDoc value = codecs.PyUnicode_DecodeRawUnicodeEscape(line,
1742                                                                   "strict");
1743            push(new PyString(value));
1744        }
1745
1746        final private void load_binunicode() {
1747            String JavaDoc d = file.read(4);
1748            int len = d.charAt(0) |
1749                     (d.charAt(1)<<8) |
1750                     (d.charAt(2)<<16) |
1751                     (d.charAt(3)<<24);
1752            String JavaDoc line = file.read(len);
1753            push(new PyString(codecs.PyUnicode_DecodeUTF8(line, "strict")));
1754        }
1755
1756        final private void load_tuple() {
1757            PyObject[] arr = new PyObject[marker()];
1758            pop(arr);
1759            pop();
1760            push(new PyTuple(arr));
1761        }
1762
1763        final private void load_empty_tuple() {
1764            push(new PyTuple(Py.EmptyObjects));
1765        }
1766
1767        final private void load_empty_list() {
1768            push(new PyList(Py.EmptyObjects));
1769        }
1770
1771        final private void load_empty_dictionary() {
1772            push(new PyDictionary());
1773        }
1774
1775
1776        final private void load_list() {
1777            PyObject[] arr = new PyObject[marker()];
1778            pop(arr);
1779            pop();
1780            push(new PyList(arr));
1781        }
1782
1783
1784        final private void load_dict() {
1785            int k = marker();
1786            PyDictionary d = new PyDictionary();
1787            for (int i = 0; i < k; i += 2) {
1788                PyObject value = pop();
1789                PyObject key = pop();
1790                d.__setitem__(key, value);
1791            }
1792            pop();
1793            push(d);
1794        }
1795
1796
1797        final private void load_inst() {
1798            PyObject[] args = new PyObject[marker()];
1799            pop(args);
1800            pop();
1801
1802            String JavaDoc module = file.readlineNoNl();
1803            String JavaDoc name = file.readlineNoNl();
1804            PyObject klass = find_class(module, name);
1805
1806            PyObject value = null;
1807            if (args.length == 0 && klass instanceof PyClass &&
1808                        klass.__findattr__("__getinitargs__") == null) {
1809                value = new PyInstance((PyClass)klass);
1810            } else {
1811                value = klass.__call__(args);
1812            }
1813            push(value);
1814        }
1815
1816
1817        final private void load_obj() {
1818            PyObject[] args = new PyObject[marker()-1];
1819            pop(args);
1820            PyObject klass = pop();
1821            pop();
1822
1823            PyObject value = null;
1824            if (args.length == 0 && klass instanceof PyClass &&
1825                        klass.__findattr__("__getinitargs__") == null) {
1826                value = new PyInstance((PyClass)klass);
1827            } else {
1828                value = klass.__call__(args);
1829            }
1830            push(value);
1831        }
1832
1833        final private void load_global() {
1834            String JavaDoc module = file.readlineNoNl();
1835            String JavaDoc name = file.readlineNoNl();
1836            PyObject klass = find_class(module, name);
1837            push(klass);
1838        }
1839
1840
1841        final private PyObject find_class(String JavaDoc module, String JavaDoc name) {
1842            PyObject fc = dict.__finditem__("find_global");
1843            if (fc != null) {
1844               if (fc == Py.None)
1845                   throw new PyException(UnpicklingError,
1846                         "Global and instance pickles are not supported.");
1847               return fc.__call__(new PyString(module), new PyString(name));
1848            }
1849
1850            PyObject modules = Py.getSystemState().modules;
1851            PyObject mod = modules.__finditem__(module.intern());
1852            if (mod == null) {
1853                mod = importModule(module);
1854            }
1855            PyObject global = mod.__findattr__(name.intern());
1856            if (global == null) {
1857                throw new PyException(Py.SystemError,
1858                          "Failed to import class " + name + " from module " +
1859                          module);
1860            }
1861            return global;
1862        }
1863
1864
1865        final private void load_reduce() {
1866            PyObject arg_tup = pop();
1867            PyObject callable = pop();
1868            if (!(callable instanceof PyClass)) {
1869                if (safe_constructors.__finditem__(callable) == null) {
1870                    if (callable.__findattr__("__safe_for_unpickling__")
1871                                                              == null)
1872                        throw new PyException(UnpicklingError,
1873                                callable + " is not safe for unpickling");
1874                }
1875            }
1876
1877            PyObject value = null;
1878            if (arg_tup == Py.None) {
1879                // XXX __basicnew__ ?
1880
value = callable.__findattr__("__basicnew__").__call__();
1881            } else {
1882                value = callable.__call__(make_array(arg_tup));
1883            }
1884            push(value);
1885        }
1886
1887        final private PyObject[] make_array(PyObject seq) {
1888            int n = seq.__len__();
1889            PyObject[] objs= new PyObject[n];
1890
1891            for(int i=0; i<n; i++)
1892                objs[i] = seq.__finditem__(i);
1893            return objs;
1894        }
1895
1896        final private void load_pop() {
1897            pop();
1898        }
1899
1900
1901        final private void load_pop_mark() {
1902            pop(marker());
1903        }
1904
1905        final private void load_dup() {
1906            push(peek());
1907        }
1908
1909        final private void load_get() {
1910            String JavaDoc py_str = file.readlineNoNl();
1911            PyObject value = (PyObject)memo.get(py_str);
1912            if (value == null)
1913                throw new PyException(BadPickleGet, py_str);
1914            push(value);
1915        }
1916
1917        final private void load_binget() {
1918            String JavaDoc py_key = String.valueOf((int)file.read(1).charAt(0));
1919            PyObject value = (PyObject)memo.get(py_key);
1920            if (value == null)
1921                throw new PyException(BadPickleGet, py_key);
1922            push(value);
1923        }
1924
1925        final private void load_long_binget() {
1926            String JavaDoc d = file.read(4);
1927            int i = d.charAt(0) |
1928                   (d.charAt(1)<<8) |
1929                   (d.charAt(2)<<16) |
1930                   (d.charAt(3)<<24);
1931            String JavaDoc py_key = String.valueOf(i);
1932            PyObject value = (PyObject)memo.get(py_key);
1933            if (value == null)
1934                throw new PyException(BadPickleGet, py_key);
1935            push(value);
1936        }
1937
1938
1939        final private void load_put() {
1940            memo.put(file.readlineNoNl(), peek());
1941        }
1942
1943
1944        final private void load_binput() {
1945            int i = (int)file.read(1).charAt(0);
1946            memo.put(String.valueOf(i), peek());
1947        }
1948
1949
1950        final private void load_long_binput() {
1951            String JavaDoc d = file.read(4);
1952            int i = d.charAt(0) |
1953                   (d.charAt(1)<<8) |
1954                   (d.charAt(2)<<16) |
1955                   (d.charAt(3)<<24);
1956            memo.put(String.valueOf(i), peek());
1957        }
1958
1959
1960        final private void load_append() {
1961            PyObject value = pop();
1962            PyList list = (PyList)peek();
1963            list.append(value);
1964        }
1965
1966        final private void load_appends() {
1967            int mark = marker();
1968            PyList list = (PyList)peek(mark+1);
1969            for (int i = mark-1; i >= 0; i--)
1970                list.append(peek(i));
1971            pop(mark+1);
1972        }
1973
1974        final private void load_setitem() {
1975            PyObject value = pop();
1976            PyObject key = pop();
1977            PyDictionary dict = (PyDictionary)peek();
1978            dict.__setitem__(key, value);
1979        }
1980
1981
1982        final private void load_setitems() {
1983            int mark = marker();
1984            PyDictionary dict = (PyDictionary)peek(mark+1);
1985            for (int i = 0; i < mark; i += 2) {
1986                PyObject key = peek(i+1);
1987                PyObject value = peek(i);
1988                dict.__setitem__(key, value);
1989            }
1990            pop(mark+1);
1991        }
1992
1993        final private void load_build() {
1994            PyObject value = pop();
1995            PyInstance inst = (PyInstance)peek();
1996            PyObject setstate = inst.__findattr__("__setstate__");
1997            if (setstate == null) {
1998                inst.__dict__.__findattr__("update").__call__(value);
1999            } else {
2000                setstate.__call__(value);
2001            }
2002        }
2003
2004        final private void load_mark() {
2005            push(mark);
2006        }
2007
2008        final private PyObject load_stop() {
2009            return pop();
2010        }
2011
2012
2013
2014        final private PyObject peek() {
2015            return stack[stackTop-1];
2016        }
2017
2018        final private PyObject peek(int count) {
2019            return stack[stackTop-count-1];
2020        }
2021
2022
2023        final private PyObject pop() {
2024            PyObject val = stack[--stackTop];
2025            stack[stackTop] = null;
2026            return val;
2027        }
2028
2029        final private void pop(int count) {
2030            for (int i = 0; i < count; i++)
2031                stack[--stackTop] = null;
2032        }
2033
2034
2035        final private void pop(PyObject[] arr) {
2036            int len = arr.length;
2037            System.arraycopy(stack, stackTop - len, arr, 0, len);
2038            stackTop -= len;
2039        }
2040
2041        final private void push(PyObject val) {
2042            if (stackTop >= stack.length) {
2043                PyObject[] newStack = new PyObject[(stackTop+1) * 2];
2044                System.arraycopy(stack, 0, newStack, 0, stack.length);
2045                stack = newStack;
2046            }
2047            stack[stackTop++] = val;
2048        }
2049    }
2050
2051
2052    private static PyObject importModule(String JavaDoc name) {
2053        PyObject silly_list = new PyTuple(new PyString[] {
2054            Py.newString("__doc__"),
2055        });
2056        return __builtin__.__import__(name, null, null, silly_list);
2057    }
2058
2059    private static PyObject getJavaFunc(String JavaDoc name) {
2060        return Py.newJavaFunc(cPickle.class, name);
2061    }
2062
2063    private static PyObject buildClass(String JavaDoc classname,
2064                                       PyObject superclass,
2065                                       String JavaDoc classCodeName,
2066                                       String JavaDoc doc) {
2067        PyObject[] sclass = Py.EmptyObjects;
2068        if (superclass != null)
2069             sclass = new PyObject[] { superclass };
2070        PyObject cls = Py.makeClass(
2071                            classname, sclass,
2072                            Py.newJavaCode(cPickle.class, classCodeName),
2073                            new PyString(doc));
2074        return cls;
2075    }
2076}
2077
Popular Tags