I have an email account whose sole purpose it is to store interesting and useful links to programming articles, code, and blog posts. It has become a little knowledgebase of sorts. I can even do a search on it, which is pretty cool.
However, after using this account for a couple of years, I now have 775 links, and it has become this big blob of amorphous information, most of which I have never looked at again. I take comfort in the fact that, if I really needed to, I could find something in there again, if I even remember putting it in there in the first place. But it has developed a "smell," if you will.
How are you organizing your programming library of cool stuff? Do you have a system or tool, and is it better than the way I'm doing it?
I would use something that is made for storing bookmarks. I use delicious.com for all of my bookmarks. The tagging system works perfectly for technology sites because you can tag each page with a specific language or tech abbreviation. This coupled with the Delicious Bookmarks plugin will make it very easy to tag sites and get back to them.
Use one word or abbreviations for languages: java c# vb.net python
Use acronyms for technologies: wpf wcf
I used to use the standard bookmark system in the browser but since I bounce around through various machines and browsers throughout the day I started to use bookmark synchronizers. Both Foxmarks and the one that google came out with. But neither I was completely satisfied with. Plus delicious has a great web interface to it as a decent api to extend for your own purpose.
IMHO, using Evernote to store this information is great.
1) you can go back and search through it easily
2) organize by tags and "notebooks collections"
3) available on multiple platforms (even mobile)
4) available as browser plugins (for direct archiving in-browser)
The only drawback is it's copy-paste functionality is a little lacking (it sometimes doesn't import/display the CSS styles correctly).
Otherwise, it's a great alternative to store web "bookmarks" (and also archive the content at the same time).
Related
I'm looking to scrape public data off of many different local government websites. This data is not provided in any standard format (XML, RSS, etc.) and must be scraped from the HTML. I need to scrape this data and store it in a database for future reference. Ideally the scraping routine would run on a recurring basis and only store the new records in the database. There should be a way for me to detect the new records from the old easily on each of these websites.
My big question is: What's the best method to accomplish this? I've heard some use YQL. I also know that some programming languages make parsing HTML data easier as well. I'm a developer with knowledge in a few different languages and want to make sure I choose the proper language and method to develop this so it's easy to maintain. As the websites change in the future the scraping routines/code/logic will need to be updated so it's important that this will be fairly easy.
Any suggestions?
I would use Perl with modules WWW::Mechanize (web automation) and HTML::TokeParser (HTML parsing).
Otherwise, I would use Python with the Mechanize module (web automation) and the BeautifulSoup module (HTML parsing).
I agree with David about perl and python. Ruby also has mechanize and is excellent for scraping. The only one I would stay away from is php due to it's lack of scraping libraries and clumsy regex functions. As far as YQL goes, it's good for some things but for scraping it really just adds an extra layer of things that can go wrong (in my opinion).
Well, I would use my own scraping library or the corresponding command line tool.
It can use templates which can scrape most web pages without any actual programming, normalize similar data from different sites to a canonical format and validate that none of the pages has changed its layout...
The command line tool doesn't support databases through, there you would need to program something...
(on the other hand Webharvest says it supports databases, but it has no templates)
I'm in the process of quoting out a website build for a law firm.
I was hoping you could give me your opinions on what the best CMS system would be to easily achieve what they want.
It's a basic informational website. News, About Us, List of Attorneys, Attorney Bios, etc. They would like to be able to easily edit all of these sections.
I was looking into Joomla as a CMS solution to allow them to do this, but I am unsure that it will get the job done (keep in mind I don't have much experience with it however). Joomla seems to revolve around "Articles". This would be fine for the News section, but when it comes to the List of Attorneys (which would have name, location, contact) and would need to be sortable, etc. I don't get the feeling it would get the job done. This could easily be my lack of understanding however.
With a site like this, what do you think the best back-end system would be for them to update this type of content?
This sounds like a rather standard website.
Most CMS-es, open source or proprietary, will solve your usecase rather easily. It really boils down to what tools you are familiar with and if you have special needs.
Of open source CMS-es Plone will easily solve your challenges, and if you prefer php-based solutions, joomla, drupal or wordpress will probably do so as well.
There is a comparisation of their respective strengths and disadvantages available here:
http://www.idealware.org/reports/2010-os-cms
As a developer who's made more than a couple very successful Joomla websites I can definitely say you're misunderstanding 'articles' in this case.
You can have an 'About Us' page, a 'Contact Us' Page, any page you want. Each 'page' is just listed as an 'article' in the backend to make it easy for someone who doesn't know the system to have an idea where to look to edit that 'page'.
In terms of 'lists' of things - there are so many great extensions already written for Joomla you would be really hard pressed to find a better CMS that will allow you to get exactly what you're looking for as quickly or as easily. K2 is great for certain situations but I don't think it would fit the bill for this project.
What you mentioned seems like a standard Joomla install honestly, straight out of the box. A few pages and a list of attorneys? No sweat. You might even look into the Sobi2 component for Joomla - a fully customizable component originally designed as a 'business index' that would probably fit the bill almost perfectly for a 'list of attorneys' and could be as large, or as small as you want. I use it on one of my sites and in that 'list' of providers we have probably 250+ listings from all over the US. However, I'd only recommend it if it's going to get some use - otherwise you're probably fine just using the Joomla system without any plugin.
It sounds like you read a quick overview of Joomla and made a decision without having actually looked into what it can do, or looked at the demo sites created with it. Check it out again, and take a deeper look.
In fact, I had an attorney contact me relating to a Joomla site, and I had him a site up and running within about 4 hours. He had a template he wanted used, so I just installed the template, put in the text he wanted in the particular pages (i.e. articles) and it was done!
Have a look at the content component K2 for Joomla : http://getk2.org/
It allows you to easily create some kind of "pattern of articles", with additional informations, fields,..., then cou can sort them,...
Look here: http://php.opensourcecms.com/ There is a big list of great OpenSource CMS Software, choose the one which has the features which you need.
I don't have a lot of Joomla experience, so can't speak to that; but SharePoint "revolves" around lists, so it will fit your requirements pretty well.
What are the best open source image gallery engines? Both stand-alone, and for existing frameworks such as Wordpress or Drupal.
Hopefully we can build a good list here over time.
Gallery is the classic choice. It has skins, security layers, heaps of plugins, etc, but can be run with the default settings easily if you want to. I've used it for years.
GOOD QUESTION, lots of people ask this in many web forums so hopefully we will get some good responses to this, and have a good list of solutions.
Personally I always used to say something like Gallery or some other OS script, but recently I have found myself using more and more something like a simple php script which just spits our a list of images (maybe 7 a page) but relying on a Javascript library such as mootools or Ext to provide all the functionality, particularly for small or individual galleries. Im particularly loving the noobslide mootools class at the moment which has some lovely gallery effects.
Noobslide
I suppose at the end of the day its all down to what you need, there will be no one answer that fits all but a number of different solutions will hopefully show up here that will suit different peoples needs.
We are developing a web application which is available in 3 languages.
There are these key-value pairs to translate everything. At this moment we use Excel (key, german, french, english) for this. But this does not work well ... if there is more than 1 person editing this file, you have no chance to automatically merge the different files.
Is there a good (and free) tool which can handle this job?
--- additional information ---
(This is a STRUTS application) But the question is how to manage these kinds of information in general (or at least in an conveinient way, which also supports multiple users editing this single file ("mergeable" filetypes))
Why not use gettext and manage separate .po files? See that blog entry.
If you can store this information in plain text then you will be able to use a version control system like subversion to help you with merging changes. Subversion is free.
The free guide (the "Red Book") to subversion gives a fairly good explanation of how this kind of merging works.
http://svnbook.red-bean.com/en/1.5/svn.basic.vsn-models.html#svn.basic.vsn-models.copy-merge
EDIT: Another thought - if you really want to stay using a spreadsheet - Google Docs supports simultaneous editing of a spreadsheet. You could import your existing spreadsheet and get your multi-user merging wishes for free with very little change to how you work.
Good Question.
There are some "Best Practice" depending on what you actually code in (java, ms-windows c#).
I solved this (but I think there must be a better way) by using a SQL db instead of excel file, and a wrote a plug for VS (VB6,........,..., emacs) that was able to insert new keys into the db without going to round trip with version control. The keys are the developers name of what they think is a best guess for a label. (key => save, sv => "spara", no => "", en => "save").
This db can then be generated as a module, class, obj, txt, to appropriate code(platform)
and can be accessed, depending on the ide, so in c#, bt,label = corelang.save;
Someone else can then do all the language stuff, and then we just update the db and rerun the generation to the platform resources.
After years of seeing localization done, including localization at large companies like Sony. I can only say the "standard" is Excel :)
There are tons of good ideas around, and probably many better ways to do it, but in real-life excel seems to be the best/cost effective solution that doesn't require training or making complex new tools to get the job done.
Found out, that Intellij Idea (at leas in version 7 and 8) has an editor for application resources. But it is not free at all. And it does not scale for bigger resource files with more than 1.000 keys.
Another good choice would be to use Google's spreadsheets ... for those who don't know it - it is like an "online Excell web-application". It can handle concurrent access from multiple users. Yay! But sadly, it comes from Google. This makes it impossible to be used in commercial projects.
So,
still searching...
cheers,
mana
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
So I assume I'm not the only one. I'm wondering if there are others out there who have compiled a personal code library. Something that you take from job to job that has examples of best practices, things you are proud of, or just common methods you see yourself using over and over.
I just recently started my C# library. It already has quite a few small items. Common Regex validations, interfaces for exception handling, some type conversion overloads, enum wrappers, sql injection detection methods, and some common user controls with AJAX toolkit examples.
I'm wondering what kind of things do you have in yours?
I use my own wiki where I post code snippets and commentaries.
I find that more useful than having my own library. And since they are essentially notes and not full programs there isn't a problem with who owns the code (you or your employer ).
PS: I don't hide the fact that I have that from my employer. In fact most of them were positive and even asked for a copy.
Because I primarily do web development, I've abstracted out some common features that I end up doing frequently on sites for clients.
Ajax Emailer. Nearly every site I work on has some type of contact form. I wrote a utility that allows me to drop some HTML on a page, having JavaScript field validation, and a PHP library that requires me to change a few parameters to work with each client's mail server. The only thing I have to write is CSS each time I include it on to a page.
Stylesheet skeleton generator. I wrote a small JavaScript utility that walks the DOM for whatever page it has been included on and then stubs out a valid CSS skeleton so that I can immediately start writing styles without having to do the repetitive task for every site I work on.
JavaScript Query String Parser. Occasionally I need to parse the query string but it doesn't warrant any major modifications to the server (such as installing PHP), so I wrote a generic JavaScript utility that I can easily configure for each site.
I've got other odds and end utilities, as well, but they are kind of hacked together for personal use. I'd be embarrassed to let anyone see the source.
Update
Several people have asked for my stylesheet skeleton generator in the comments so I'm providing a link to the project here. It's more or less based on the way that I structure my XHTML and format my CSS, but hopefully you'll find it useful.
I have found that using Snipplr makes this incredibly convenient. You can tag items, save favorites, search by keyword, etc. I mostly use it for Vim-related snippets (common commands, vimrc file, etc.), but it can be used for anything. Check it out.
I have my personal C++ cross platform library here: http://code.google.com/p/kgui/
It's open source LGPL, I use it in my hobby / volunteer projects. I started it about 3 years ago and have been slowly adding functionality to it.
Back in the days of C programming on MacOS 7, i did write a fairly extensive OO library (yes, OOP in very old C) mostly to handle dialog windows. I abandoned it for PowerPlant (a nice C++ from Metrowerks) during the switch from 68k to PPC processors.
A little after that, i began writing web apps, first in PHP, recently in Django. On this aspect, my reusable code is limited to some tricks and code style.
But for all non-web (or with only small web componets), i've been using Lua. It's so fast to write and rewrite code, that there's very little incentive in reusing code. I mean, what's the point of copying a 10 line function and then adapt it? it's faster to rewrite it just for this project.
That's not so wasteful as it sounds. Lua code is so succint that my apps can be very complex, but seldom have more than a couple thousands lines.
At the same time, several Lua projects imply interfacing to C libraries. It's very easy to write bindings to existing libraries, so i just do that as a subproject. And these modules are what i do reuse! once and again... with very little (if any) changes from one project to the other.
In short: non-web projects are usually one-off Lua code, and some heavily reused binding modules.
I use Source Code Library from http://www.highdots.com/products/source-code-library/ since I can manage different textfiles, notes, screenshots and different programming languages.
I have several utility MATLAB functions that I have taken with me as I move from job to job, particularly ones that enforce W3C standards on the plots I make to ensure that text and background colors have a good luminosity ratio. I also have a function that uses ActiveX to insert a MATLAB figure into PowerPoint.
I keep my personal code libraries on CPAN. I'm not even sure how I'd do this in other languages anymore. It's just too integrated in the way that I think about programming now.
For my PHP work I started with a small file of simple things: a mail function that checks inputs for header attacks, and email validator, an input srubber, that type of thing. Over time it has grown into a application framework for quickly developing one off applications that can be templated by our graphic designer.
I have a library that i use quite extensively. I started fresh with c# and kinda threw all of the legacy stuff out the window. I find them very handy and i rewrite/refactor them often (some of them). Some of the stuff i have is:
Auxiliary (things like IsRunningLocal, InternetDetection)
Standard Classes or Structs for: Address, CreditCard, Person
I have .dll's for both win and web stuff, some very logical like a .dll for shopping cart stuff
I wrote a quick and simple library in Java which I can add code snippets to. I plan to extend it to a full framework for development at some point but only when time allows. I have all sorts in there from simple functions to full blown pages and features. Its so helpful to have when developing because as a web designer, all I need to do is change the CSS of the page.