In the kindle particular case it's not private at all, amazon owns the software and the only store (it is possible sideload books with calibre), it updates automatically, phones home all the time with no control or knowledge of what is transmitting.
There used to be sony e-readers but they got discontinued and other alternatives to kindle (like nook or kobo) also have their own stores and are designed to be connected and sync whenever possible.
One alternative is to leave the device in airplane mode and load the books through usb only.