GeezerVison is showing github.com — Add support for multiple sources · Issue #84 · contentlayerdev/contentlayer
Add support for multiple sources #84DescriptionCurrently, Contentlayer looks for a single exported method that is pulled from one particular source. We've received requests to support more than one source within the same configuration. I'm working to get more information on specific use cases.
Considerations
What are the challenges with this approach?
Design
What would the API for this look like? Perhaps you could define sources separately and then reference the source at the document level? And include the option for a default source.
Napkin sketch:
const ctfl = makeSource({ type: 'contentful', accessToken: /* ... */ })
const local = makeSource({ type: 'local', path: /* ... */ })
const Page = defineDocumentType({ source: ctfl })
const Post = defineDocumentType({})
export default {
defaultSource: local,
documentTypes: [Page, Post]
}Reacted by 684efs3, Jacob Marshall, Joe Bell, David, WangJie, Lachlan Campbell, Arrofi Reza Satria, Gajendrasingh Dawar, Johnny Chai, Katsuma Ito and 10 more Activityseancdavis commented on Jan 5, 2022 CollaboratorAuthor More actions We're looking for more examples to better understand the use case before introducing this feature. timlrx commented on Jan 8, 2022 I am looking to use it for md/mdx content from possibly different contentDirPath with different mdx plugins. The API looks mostly good to me. Not sure what defaultSource is meant to do though. To make it a little cleaner how about requiring the user to be more explicit on the type in makeSource.
Example types would include: markdown, mdx or contentful with their respective options. If there are plans to support non-local md or mdx, path could also take the form of an s3 URL path. Reacted by Johannes Schickling 684efs3 commented on Feb 20, 2022 When do you plan on releasing this?
In terms of specific use cases, I have mdx files in multiple files, one for a blog, another for news, etc. seancdavis commented on Feb 23, 2022 CollaboratorAuthor More actions @684efs3 We're not actively working on this feature at this time, though still considering it and looking for more use cases to determine how to shape the API.
In this case, "multiple sources" means that your collection of data is coming from multiple sources (local, headless CMS, API, etc.).
In your case, it sounds like all your data is local, but spread across multiple files. This is what Contentlayer is designed to solve today. You can create an individual document type for content in each directory. You can even process some content types as MDX and others as plain markdown. Reacted by 684efs3, Tom Byrer, Gijs Nelissen and Johnny Chai 684efs3 commented on Feb 23, 2022 Ah I see! Apologies! Your comment is very helpful. I'm looking forward to testing it! Thank you very much! :-) Reacted by Sean C Davis and Johnny Chai cjroebuck commented on May 27, 2022 @schickling @seancdavis I'd like to ask if this feature would make it possible to read in content sources from a remote git(hub) repo, a bit like how nextjs does it for their own docs site (https://github.com/vercel/next.js/tree/canary/docs)
Also see (https://github.com/opstrace/next-product-docs).
This means, for example, that we could then keep our monorepo private, but have a public repo that includes just the markdown content, that anyone can request to make change in, such as typos etc.
I guess it might get tricky when using components inline in the MDX that would be from the private repo, but maybe not.
I just figured this is possible now using something like git submodules to link the public repo into the private repos content directory seancdavis commented on May 28, 2022 CollaboratorAuthor More actions @cjroebuck I like the pattern you're suggesting a lot. And yes, your last two points are important ones:
- Submodules are a good solution for bringing in content from another repo and using Contentlayer to load that content into your private site.
- There is an inherent risk in specifying functionality (i.e. using components) in a content project that shouldn't know anything about its front ends.
This issue is specifically about using multiple sources in the same project — e.g. some content comes from markdown files, other from an API like Contentful. ahofmeister commented on Sep 17, 2023 Another use case would be to separate blog entries from projects for a personal portfolio or is there any other solution to implement it? phyesix commented on Sep 27, 2023 Is there any progress on this? My old blog's posts are md and I write my current posts with mdx. Can we write both md and mdx to filePathPattern in contentlayer.config (ex: filePathPattern: [**/*.md, **/*.mdx])? ahofmeister commented on Sep 28, 2023 @phyesix Yes, it is possible to write both. :) What I have done is the following:
Create multiple document types and refer them in the makeSource method like
export default makeSource({
contentDirPath: 'src/content',
documentTypes: [Post, Project],
mdx: {
remarkPlugins: [remarkGfm],
rehypePlugins: [
[
rehypePrettyCode, options
],
],
},
})
Then you can just create the document type like this
export const Project = defineDocumentType(() => ({
name: 'Project',
filePathPattern: `**/projects/*.mdx`,
contentType: 'mdx',
...
...
...
This allows to have different file types. You could just use *.md and *.mdx. Reacted by Hunter Garrett, Carlo Taleon and Abdullah Blankeos commented on Nov 11, 2023
Create multiple document types and refer them in the makeSource method like
@ahofmeister this works like a charm for me. I structured my folders like the following
> src
> content
> posts
- *.mdx
> projects
- *.mdx
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|