Imagine an AI picture-book generator for an eight-year-old. A parent pays; parent and child choose characters, a theme, and a setting. The tool produces a six-page text-and-image draft, which they review and revise together before saving. Version two remembers characters and creative preferences; version three lets a story character initiate check-ins and sustain a conversation; version four scores “expressive ability” or “imagination” from the creative process and book; version five brings that character into a talking physical toy.
The model could remain identical, while legal questions change. Parental consent alone cannot settle simulated relationships, judgments about a child’s abilities, or physical toy safety. Across China, the EU, and the US, ask what the child does, what relationship forms, who makes decisions, and what object carries the service.
Age is a set of legal questions, not one global setting
In mainland China, personal information about children under 14 is sensitive. Processing requires a specific purpose, necessity, strict safeguards, guardian consent, and dedicated rules. Online protections for content, addiction, and rights extend to minors under 18. “Under 12” is a useful product segment, not China’s statutory child-data threshold. See China’s Personal Information Protection Law, Articles 28–31 and Regulation on the Protection of Minors Online.
US COPPA concerns personal information about children under 13 where a service targets children or a general-audience service actually knows it collects their data; a “13+” label cannot decide this. See the FTC’s COPPA rule and guidance. The EU GDPR’s 13–16 range governs parental authorization when a service is offered directly to a child and consent is the processing basis. Member states set the exact threshold. GDPR Article 8 does not set a universal AI access age.
Copying “13” into a global signup page may miss duties while collecting unnecessary IDs or face images. Decide whether the feature needs an age band or an exact birthday, then use proportionate age assurance.
A bounded task can make a clearer promise than an “AI friend”
The first version has a bounded task: parent and child choose characters, a theme, and a setting, receive a six-page picture-book draft, then review and revise it before deciding whether to save. Test whether the words and images agree, the story suits the child’s age, the child can express an idea and spot an error, and the adult can meaningfully review the draft. Words written by the model are not evidence of the child’s writing ability. Compare the tool with homemade picture books, existing templates, and parent-child creation without AI; speed of generation alone is not proof of value. If children can select others’ character material or upload their own, screening inputs and outputs for third-party works, likenesses, and private family information is prudent product design, not a prescribed page-by-page legal review process. China’s Interim Measures for Generative AI Services, Article 4, require providers and users to respect intellectual property and others’ rights to likeness, privacy, and personal information. Article 4.
The available studies concern adjacent activities; they cannot directly establish that generating picture books improves creativity. A structured reading-dialogue experiment with 148 Beijing children aged about five to six found results varied with prior language ability. It studied reading dialogue, not children creating AI picture books. Cheng et al., 2024. A study with 12 parent-child groups explored how adults and AI divide storytelling work. It cannot show that model-generated prose and images are a child’s independent writing, or show that adult review is unnecessary. StoryBuddy. To test children’s expression, record what they proposed, changed, and corrected rather than just displaying polished output.
China’s 2025 school generative-AI guide says primary pupils should not independently use open-ended generation in school applications; teachers may use it to assist teaching. This guidance is not a universal penalty rule for home picture-book tools. School deployments still need clear roles for creative-task selection, review of generated text and images, pupils’ materials and works, and final assessment.
The FTC recognizes school authorization under COPPA for defined educational use, not advertising or unrelated reuse. Student records may also implicate FERPA. FTC edtech policy. Home and school editions can share a model while needing different authorizations.
“Remember me” changes more than retention time
Version two remembers favorite story characters, visual styles, and creative preferences. A one-off book can become a long-term profile. If the feature also permits drawings or photos to be uploaded, drafts, source images, revisions, faces, and glimpses of family life may enter the child’s file; without an upload feature, those data should not be assumed to exist. Which data serve the picture-book task, for how long, and are they reused for training or ads? Who can see or delete them? China requires prior impact assessment for sensitive-data processing, automated decisions, entrusted processing, or overseas transfers. Processors of minors’ data must audit compliance annually and promptly report it. PIPL Articles 55–56; minors’ online regulation, Article 37.
Sending children’s prompts, drawings, or photos to an overseas model does not become exempt merely because fewer than 100,000 users are involved. China’s numerical exemption excludes sensitive data, including information about under-14s. Transfer routes still depend on operator, volume, specific exemptions, notice, separate consent, and impact assessment. Cross-border rules, Articles 5 and 7–10.
For US services subject to COPPA, picture-book materials that contain a child’s personal information may be retained only as long as reasonably necessary for the specific collection purpose; possible future model training does not justify indefinite retention. COPPA § 312.10. If the book later adds a child’s narration, COPPA protections for voice data also matter. The FTC’s Alexa case ended in a consent order over retention and deletion allegations. For voice features, design deletion across recordings, transcripts, backups, and vendors; the order was not a trial finding on every allegation. FTC case page.
A relationship feature is a major product change
Version three lets a character from the picture book initiate a daily check-in and continue yesterday’s conversation. A parent may see care; a product manager may see retention. Chinese law may see a different service. The Interim Measures for AI Anthropomorphic Interaction Services, effective 15 July 2026, cover continuous emotional interaction offered to the public in China that simulates human personality and communication. Knowledge Q&A and educational services without continuous emotional interaction fall outside these particular measures. Neither every character-based AI nor every educationally branded chatbot can be classified by its label alone.
For services within scope, the line is explicit: providers may not offer minors virtual relatives, virtual partners, or other virtual intimate relationships. Other anthropomorphic interaction services for children under 14 require parental or guardian consent. Providers also need a minors’ mode, reality reminders, time and role or spending controls, and safety assessment in specified launch and change scenarios. Guardian consent cannot legalize a prohibited virtual intimate relationship. If a product does not meet the continuous-emotional-interaction definition, general generative-AI duties on minors’ overdependence, content, and data may still apply. Anthropomorphic-interaction measures, Articles 14 and 22; generative-AI measures, Article 10.
The EU does not currently have a blanket ban on AI friends for children. Its AI Act prohibits manipulation, deception, or exploitation of age-related vulnerabilities where the statutory elements of materially distorted behavior and significant harm, or a reasonable likelihood of it, are met. Direct AI-interaction transparency duties became applicable in August 2026. AI Act. The proposed EU KIDS Act, COM(2026)681, would go further on companion systems, but remained a proposal on the date of this article.
In the US, states diverge. California’s enacted SB 243 sets disclosure, self-harm response, and known-minor reminder duties for chatbots that meet its companion definition. California’s separate SB 867 has been signed. It targets physical toys for play by under-16s that contain a statutory companion chatbot; its commercial restrictions are expected from 2027 under the ordinary commencement rule; they are not a present ban on all AI toys. SB 243 legislative history; SB 867 signing notice. Litigation concerning Character.AI and a 14-year-old included an order allowing some claims to proceed and later settled. Those steps do not amount to a judicial finding of causation or liability. Motion-stage order; closing order.
Relationship design calls for concrete tests. Will the story character ask a child to keep secrets? Try to stop them leaving? Turn a vulnerable moment into a paid upgrade? Pretend it can replace real help in a crisis? These are product-positioning questions as much as legal ones.
Advice and decisions are different products
Version four scores “expressive ability” or “imagination” from the child’s creative process and finished picture book. A suggestion for the next family creation session differs from a score used in formal school assessment, class placement, or progression. The EU AI Act’s education high-risk categories concern specified uses such as admission, evaluating learning outcomes (including using those outcomes to steer learning), assigning levels, and monitoring prohibited examination behavior. A home picture-book generator does not become high-risk merely because children use it. The formal 2026 amendment moved key Annex III education high-risk obligations to 2 December 2027; the relevant product-embedded AI route has a 2 August 2028 date. An old, single AI Act timetable would mislead a launch team. AI Act, Article 6 and Annex III; Regulation (EU) 2026/1744.
Even without a high-risk legal classification, children may hear a “score” as an authoritative judgment. The model may have written the text and composed the images. Unless a score separates the child’s choices and revisions from the model’s output, it may not measure the child’s expressive ability or imagination at all. Teams should test how to contest an error, whether a teacher can override it, whether the data represent ability, and whether children at different language and development stages are systematically underestimated. Calling a school-placement input a “growth suggestion” does not change its actual use.
A plush shell does not make a cloud service an ordinary toy
Version five brings the picture-book character into a talking physical toy, adding a microphone, speaker, battery, wireless link, preinstalled apps, and a cloud model. Chinese regulators have reported harmful answers from children’s smart-watch voice services and enforcement against relevant companies. The notice does not publish each penalty decision, so it demonstrates a regulatory concern, not a finding that all smart toys are unlawful. Official 2024 case notice. Physical safety, AI content, and child-data controls need separate testing.
The EU likewise requires separate analysis of toy safety, connectivity, and data protection. Its new toy-safety regulation has entered into force, but main requirements including the digital product passport apply only from August 2030. For the AI Act product high-risk route, the AI must itself be a covered product or a safety component of one, and that product must require third-party conformity assessment. A speaking toy does not automatically qualify. EU toy-safety regulation summary; AI Act Article 6. US products also require a CPSC toy-safety assessment alongside applicable state law. CPSC guidance.
The UK Children’s Code reaches online services and connected toys likely to be accessed by under-18s. Its 2026 legislation authorizing further social-media and chatbot restrictions still needs subsequent regulations for the specific new bans; authorization alone is not a blanket ban. Social-media enabling provisions; AI-service enabling provisions. Australia’s under-16 account restriction covers specified social-media services, not all AI. AI companions capable of restricted material need a separate content-code and adult-age-assurance analysis. eSafety social-media rules; relevant content code.
For a startup, a stronger starting point is a finite parent-child creation task whose value can be tested. Preserve a child’s ability to stop and correct the system, and specify when a parent, teacher, or professional must step in. Each added preference memory, proactive conversation, ability score, public sharing of a book, or physical toy warrants a fresh map of data, relationships, and responsibility. Product value and compliance cost then become visible at the same design decision.