1. According to API, it's possible to implement custom renderer or layer property pages. How could we make it for text background ?
2. How to save my custom text background in a MXD ? Currently, when a custom text background is applied to features, the text symbol is lost when I save and reopen the MXD.
I don't think text is "rendered" in ArcMap, but I could be wrong. I did a search in the Java API for the word Renderer and other than ArcIMS Labels, the only renderers I saw were for features, legends, datasets or rasters.
Any of the classes that support persistence will have the writeExternal method.
Looking at TextElement, it says it will accept geometry of type esriGeometryPoint, esriGeometryPolyline, or esriGeometryBag. Maybe you could look at implementing a custom feature renderer and apply that renderer to a TextElement's symbol? Or a ParagraphTextElement's symbol?
3. I use Maplex for the placement of labels. I have set Maplex in order to labels using Line Callout do not overlap. But when I apply my custom text background, it overlap. However, in my custom text background class, I have well implemented the functions queryBoundary, getGeometry, queryEnvelope and queryMargins but they are never called when Maplex is labelling (tested in debug).