TPost is a starter project for an app that can crawl websites and make posts in different sources, for example, It can use a telegram bot to publish posts into the telegram channel.
In progress...
In progress...
Crawler extracts required content from a web page and returns posts.
Interface for crawlers, it has single method GetPosts, which returns IReadOnlyCollection<IPost>.
For example, you can implement crawler using HtmlAgilityPack.
Crawler manager takes all crawlers from DI container and refill store with posts.
Default implementations is: DefaultPostCrawlerManager.
DefaultPostCrawlerManager takes count / n posts from each registered crawler. Where count is argument of method RenewPostStore (10 by default) and n is amount of registered in DI IPostCrawler's.
Post store stores crawled posts.
Default implemetation is: ConcurrentQueuePostStore.
GetAndRemoveOne method takes one post from a store and removes it. If you want to use the persistent store, and want to save published posts. You should implement a mechanism, that will not return already published posts. (Extra column in a table like IsPublished, etc.)
Interface for post publisher.
Default implementation is CompositePostPuiblisher, which takes all IPostPublisherTransport from DI container and calls Publish method on them.
Publisher transport. Inherits from IPostPublisher. Added to make CompositePostPuiblisher work.
By default behavior, you need to implement just that interface. It will allow you to have multiple publish destinations.
Post job takes post from the store and publishes it to registered IPostPublisher by cron schedule.
Publishers used to publish posts to different destinations.
If you want to use a custom publisher, implement the IPostPublisherTransport interface. In the default implementation, it will be used by CompositePostPublisher, but you can register it as a single publisher by removing registration for IPostPublisher из IServiceCollection which occurs at TPostHostFactory.
You can find working sample in _Samples/TPost.Host.Sample folder.
This sample crawling jokes from two sites and publishing them into a console. As you can see, it just implements two crawlers and registers them in a host.
I have a real project, which works like that, and it posts jokes into @cringedot Telegram channel.
| Package Name | Version | Downloads | Description |
|---|---|---|---|
| TPost.Core |  |
 |
Core abstractions |
| TPost.Hosting |  |
 |
Hosting abstractions |
| TPost.Publishers.Telegram |  |
 |
Telegram bot publisher implementation |