"BufferWriter"

Shared Code
#1

Java has this useful “interface” io.Writer. Unfortunately, in the traditional “some of our team don’t understand OOP” manner, the original designers of java.io made it a class instead of an interface. Bad move.

This made it slightly tricky for the NIO designers to make their buffers compatible with it (e.g. CharBuffer really really ought to “implements Writer”). They could have provided some kind of kludge workaround, but … didn’t.

So now all the existing thousands of java apps that sensibly were compiled to write to Writer instances cannot output directly into a Buffer. Instead you have to do lots of unnecessary copying; e.g. if dealing with chars/Strings, you have to write to a StringWriter, and then convert that to a String and then copy the String to the CharBuffer. NB: you can’t even put the StringBuffer directly - you first have to convert to a String. Very dumb API design.

So…does anyone have a CharBufferWriter (or BufferWriter) they’d care to share? Takes a CharBuffer as input, and extends Writer to write all incoming chars directly into the CharBuffer? I’m assuming it’s such a useful class that:

  • someone’s already made it
  • …or there’s some subtle reason why it’s hard
  • …or there’s some easier way of converting efficiently which I’ve been too stupid to spot.

If not, I’ll probably (*) make one and post it.

(*) …I’m trying to efficiently interface to a lib that does lots of string manipulation and which I know is pretty slow, and trying to claw back performance. Limiting the amount of copying it does (given that it needs to output a LOT of very large strings) looks like it could help with one of the bottlenecks. Then again, if it doesn’t, I’ll be looking elsewhere…

#2

I would have thought the CharSequence stuff added in 1.4 would be the way to get a StringBuffer to talk to a CharBuffer… but I haven’t actually looked into it. Hmmm… looks like you can’t write a CharSequence to a CharBuffer – that IS dumb.

Actually are you sure you need to use StringBuffer at all? Maybe you can get away with using a CharBuffer where you were using the StringBuffer.

#3

[quote]looks like you can’t write a CharSequence to a CharBuffer – that IS dumb.
[/quote]
Yeah, pretty spectacularly, especially seeing as they only just added the CS class in order to remove all the old problems with only accepting String’s >:(.

Seems there’s lots of “missing methods” in NIO at the moment ;D. e.g. I have two outstanding bugs on ones I feel are necessary for CharBuffer / BB to do with the fact that different charsets have different bytes-per-char - and currently there is no way to keep a BB and a CB in synch (because the pointers don’t update automatically, and when you move one you have no idea how far to move the other :frowning: ).

I’m working with a 3rd party product that outputs to Writer’s, so I’m not using SB’s at all, except that there is a Sun-provided Writer that outputs to an SB, and of course SB can easily go to String, which CharBuffer does allow you to put().

#4

I’d like to point out a few things:

[*]Most people miss java.nio.channels.Channels. There’s some goodies in there.

[*]In terms of design and intent, ByteBuffer, CharBuffer, etc are to byte array, char array what ReadableByteChannel, WritableByteChannel are to InputStream and OutputStream, Reader and Writer, etc. This isn’t exact, but I’m trying to motivate an understanding of how you end up using NIO rather than an exact analogy.

[*]CharBuffer is-a CharSequence. You can wrap a CharSequence or a char array in a CharBuffer. Typically, this gets you 90% of the conversions that you want.

[*]http://www.realityinteractive.com/rgrzywinski/archives/000053.html provides some information on charsets.

#5

[quote]I’d like to point out a few things:

[*]Most people miss java.nio.channels.Channels. There’s some goodies in there.
[/quote]
True, but there’s nothing to produce a writer for a buffer there, so AFAICS no use at all in this case…?

Um, I don’t understand what you’re getting at here? As an aside, I think that your statement is patently false - IMHO the main uses of BB have nothing to do with any of that: they are about getting direct-yet-safe access to native memory. That’s what the OpenGL people use them for; that’s what we use them for when doing networking; that’s what one (theoretically, if there were fewer bugs) could use them for when doing file I/O.

I consider it no accident that the primary package - nio.* - contains the buffers, and that channels are an “add-on” package that makes use of buffers, not the other way around.

But perhaps I’m completely missing your point :).

So…? I need to write directly into a pre-existing CB; as a kludge, it would at least be nice to be able to put a CS directly into a CB - but you’re only describing how to create a NEW CB out of a CS, which is not any improvement really.

I disagree entirely. I’ve almost never found myself wanting to create fancy buffers like this, and certainly not most of the time - even Sun officially recommends you re-use buffers, which makes your referenced methods useless. So, what you usually need is methods that insert data into pre-existing buffers…at least IME.

OK, but unless I’m being stupid, there’s nothing useful there ??? Why did you provide that link?

#6

[quote]True, but there’s nothing to produce a writer for a buffer there, so AFAICS no use at all in this case…?
[/quote]
Combining my points one and two should pretty clearly define a path.

I’d like to divert from the topic for a moment here and simply say: blahblahblahh, you are the sole reason that I rarely read and contribute to this forum. I constantly feel that I have to be on the defensive when writing posts and I constantly feel that I am being torn into with your responses. As to if this is your intent, I cannot say. Since my time spent on this forum is purely voluntary, I simply do not wish to feel this way so therefore I will turn my rarely into never and bid you all adieu.

#7

[quote] Hmmm… looks like you can’t write a CharSequence to a CharBuffer – that IS dumb.
[/quote]
CharBuffer.append(CharSequence csq)

#8

To make a useful CharBufferWriter it needs to include an abstract flushBuffer method which is called when the CharBuffer is full. Unlike StringBuffer you can’t increase the size of an existing buffer. Alternatively you might create a sequence of CharBuffers, allocating a new one when the current buffer is filled.
Does either of these approaches meet your needs?

#9

I’m not at all sure what you are trying to achieve here. If you have a ByteBuffer containing encoded characters, then you use a CharsetDecoder to fill a CharBuffer with the corresponding characters. The CharBuffer is of course NOT a view on the ByteBuffer. The CharsetDecoder will adjust the positions in each buffer appropriately.

#10

You over-estimate me ;D. I simply cannot see how the two come together to solve it :(. I only wish it were as clear to me as it is to you. How else could I have phrased it to say “The things you’ve said don’t seem to me to bear any resemblance to the problem at hand; if there is a way that they do, you will actually have to explain it rather than leave it unsaid/implied”? If anything, it feels like instead of answering my initial problem you’ve created some conundrum and teasingly offered some vague pointers (leaving me clueless) rather than actually responding directly.

PS I’m sorry if my response are too robust; in this case I run into a brick wall and can’t see how your things fit together, and so I just blurt out “why?”. Being very busy, my responses were blunt - I don’t have the time for pussy-footing about - but I don’t expect that to cause anything more than minor problems. And you made some statements that are the opposite of my personal experience, so I said so. Is that so offensive? Shrug. As I always say, if anyone finds I’ve offended them more than is reasonable then they should message me privately and we can try and sort it out.

#11

Sorry, that’s a red-herring from me. The bug/RFE I was citing was where you specifically want a CB view of a BB (e.g. because you have a direct BB which you want to read from and write into without copying elsewhere, or because the BB is a buffer of different datatypes where you want to take a view as a CB, read some chars, then read some bytes, then read some chars, etc - basically, a poor man’s structs, perhaps?).

Sure, if you don’t mind having multiple separate independent copies of data around then that’s a non-issue. But I kept encountering cases where it was algorithmical desirable to operte on a single BB in-situ (nb: IIRC in my bug report I specifically stated that if I was abusing the API they should let me know, since if so it was out of blind ignorance (because of the lack of docs outlining how you ought to use NIO) - IIRC the evaluation from Sun was that the RFE sounded sensible, but it’s been a very long time since I looked at that one…).

#12

!!!

Not in my 1.4 javadocs it doesn’t!

…is this 1.5.x only?

EDIT: epithet aimed at my javadocs, not Mark, in case anyone’s wondering…

EDIT2: if so, perhaps influenced by some of the rfe’s / bugs I filed in this area years ago? That would be nice…

#13

Sorry, yes that is a 1.5 addition.

#14

Obviously the character set is the red herring because only UTF16 makes sense in this case, and the ‘character’ size is always 2 bytes.
While I appreciate the irritation at having to manually synchronize the buffer positions, I’m not at all sure what a suitable solution would be.

#15

Sorry, why does only UTF 16 make sense? I encountered those problems originally because I was routinely dealing with two charsets:

  • UTF16 (beloved of java and java-based protocols)
  • ISO 8859-1 (beloved of HTTP and other older protocols)

Oh, and having “fun” with the fact that windows preferred to use a 1-byte charset by default, and linux preferred to use a 2-byte charset. That was really disturbing for a while: you get some weird parsing failures when 1-b tries to talk to 2-b on a bi-directional protocol.

#16

I’m taking my own thread way OT here, but…surely it’s simple to keep the meta-state (pointer positions etc) of buffers and their views together?

IIRC I proposed “at the very least” an extra version of the view creation method which took an extra boolean to switch between current behaviour (only share content) and my desired behaviour (share content AND meta state), but noted that a more general solution might be desirable for other use-cases, one that would allow fine-grained control of which elements of the state were shared…

I also proposed a basic addition to Charset: a method to tell you how many bytes it used per char!

#17

CharsetDecoder has methods providing the average and maximum number of characters per byte (and CharsetEncoder has the reverse values). Given that many charsets have a variable number of characters per byte, this is all that makes sense. For example, consider a Charset which represented this encoding http://www.unicode.org/reports/tr6/.

In the case of a CharBuffer as a view of a ByteBuffer, the only encoding which makes sense is the implicit encoding of Java character arrays, namely UTF16 (as of 1.5 anyway). You cannot usefully use a CharBuffer to view a sequence of bytes which are ISO8859-1 encoded characters.

#18

in 1.4 you could do

charbuf.put(CharBuffer.wrap(csq))

where csq is a CharacterSequence, instead of the append method added in 1.5.

#19

Any idea why 1.5 added CharBuffer.append(CharSequence) instead of put(CharSequence) ???

#20

probably to be more consistant with the StringBuffer methods